{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Collaborative filtering\n",
    "-----\n",
    "\n",
    "In this example, we'll build a quick explicit feedback recommender system: that is, a model that takes into account explicit feedback signals (like ratings) to recommend new content.\n",
    "\n",
    "We'll use an approach first made popular by the [Netflix prize](http://www.netflixprize.com/) contest: [matrix factorization](https://datajobs.com/data-science-repo/Recommender-Systems-[Netflix].pdf). \n",
    "\n",
    "The basic idea is very simple:\n",
    "\n",
    "1. Start with user-item-rating triplets, conveying the information that user _i_ gave some item _j_ rating _r_.\n",
    "2. Represent both users and items as high-dimensional vectors of numbers. For example, a user could be represented by `[0.3, -1.2, 0.5]` and an item by `[1.0, -0.3, -0.6]`.\n",
    "3. The representations should be chosen so that, when we multiplied together (via [dot products](https://en.wikipedia.org/wiki/Dot_product)), we can recover the original ratings.\n",
    "4. The utility of the model then is derived from the fact that if we multiply the user vector of a user with the item vector of some item they _have not_ rated, we hope to obtain a predicition for the rating they would have given to it had they seen it.\n",
    "\n",
    "![collaborative filtering](matrix_factorization.png)\n",
    "source:[ampcamp.berkeley](http://ampcamp.berkeley.edu/big-data-mini-course/movie-recommendation-with-mllib.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Preparations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os.path as op\n",
    "\n",
    "from zipfile import ZipFile\n",
    "try:\n",
    "    from urllib.request import urlretrieve\n",
    "except ImportError:  # Python 2 compat\n",
    "    from urllib import urlretrieve\n",
    "\n",
    "# this line need to be changed:\n",
    "data_folder = '/home/lelarge/data/'\n",
    "\n",
    "ML_100K_URL = \"http://files.grouplens.org/datasets/movielens/ml-100k.zip\"\n",
    "ML_100K_FILENAME = op.join(data_folder,ML_100K_URL.rsplit('/', 1)[1])\n",
    "ML_100K_FOLDER = op.join(data_folder,'ml-100k')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start with importing a famous dataset, the [Movielens 100k dataset](https://grouplens.org/datasets/movielens/100k/). It contains 100,000 ratings (between 1 and 5) given to 1683 movies by 944 users:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if not op.exists(ML_100K_FILENAME):\n",
    "    print('Downloading %s to %s...' % (ML_100K_URL, ML_100K_FILENAME))\n",
    "    urlretrieve(ML_100K_URL, ML_100K_FILENAME)\n",
    "\n",
    "if not op.exists(ML_100K_FOLDER):\n",
    "    print('Extracting %s to %s...' % (ML_100K_FILENAME, ML_100K_FOLDER))\n",
    "    ZipFile(ML_100K_FILENAME).extractall(data_folder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other datasets, see: [Movielens](https://grouplens.org/datasets/movielens/)\n",
    "\n",
    "Possible soft to benchmark: [Lenskit](http://lenskit.org/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Data analysis and formating"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Python Data Analysis Library](http://pandas.pydata.org/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>item_id</th>\n",
       "      <th>ratings</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>196</td>\n",
       "      <td>242</td>\n",
       "      <td>3</td>\n",
       "      <td>881250949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>186</td>\n",
       "      <td>302</td>\n",
       "      <td>3</td>\n",
       "      <td>891717742</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>22</td>\n",
       "      <td>377</td>\n",
       "      <td>1</td>\n",
       "      <td>878887116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>244</td>\n",
       "      <td>51</td>\n",
       "      <td>2</td>\n",
       "      <td>880606923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>166</td>\n",
       "      <td>346</td>\n",
       "      <td>1</td>\n",
       "      <td>886397596</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  item_id  ratings  timestamp\n",
       "0      196      242        3  881250949\n",
       "1      186      302        3  891717742\n",
       "2       22      377        1  878887116\n",
       "3      244       51        2  880606923\n",
       "4      166      346        1  886397596"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "all_ratings = pd.read_csv(op.join(ML_100K_FOLDER, 'u.data'), sep='\\t',\n",
    "                          names=[\"user_id\", \"item_id\", \"ratings\", \"timestamp\"])\n",
    "all_ratings.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check out a few macro-stats of our dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100000"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#number of entries\n",
    "len(all_ratings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    100000.000000\n",
       "mean          3.529860\n",
       "std           1.125674\n",
       "min           1.000000\n",
       "25%           3.000000\n",
       "50%           4.000000\n",
       "75%           4.000000\n",
       "max           5.000000\n",
       "Name: ratings, dtype: float64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_ratings['ratings'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# number of unique rating values\n",
    "len(all_ratings['ratings'].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    100000.00000\n",
       "mean        462.48475\n",
       "std         266.61442\n",
       "min           1.00000\n",
       "25%         254.00000\n",
       "50%         447.00000\n",
       "75%         682.00000\n",
       "max         943.00000\n",
       "Name: user_id, dtype: float64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_ratings['user_id'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "943\n"
     ]
    }
   ],
   "source": [
    "# number of unique users\n",
    "total_user_id = len(all_ratings['user_id'].unique())\n",
    "print(total_user_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    100000.000000\n",
       "mean        425.530130\n",
       "std         330.798356\n",
       "min           1.000000\n",
       "25%         175.000000\n",
       "50%         322.000000\n",
       "75%         631.000000\n",
       "max        1682.000000\n",
       "Name: item_id, dtype: float64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_ratings['item_id'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1682\n"
     ]
    }
   ],
   "source": [
    "# number of unique rated items\n",
    "total_item_id = len(all_ratings['item_id'].unique())\n",
    "print(total_item_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For spliting the data into _train_ and _test_ we'll be using a pre-defined function from [scikit-learn](http://scikit-learn.org/stable/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "ratings_train, ratings_test = train_test_split(\n",
    "    all_ratings, test_size=0.2, random_state=42)\n",
    "\n",
    "user_id_train = ratings_train['user_id']\n",
    "item_id_train = ratings_train['item_id']\n",
    "rating_train = ratings_train['ratings']\n",
    "\n",
    "user_id_test = ratings_test['user_id']\n",
    "item_id_test = ratings_test['item_id']\n",
    "rating_test = ratings_test['ratings']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "80000"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(user_id_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "943"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(user_id_train.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1653"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(item_id_train.unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that all the movies are not rated in the train set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "75220    807\n",
       "48955    474\n",
       "44966    463\n",
       "13568    139\n",
       "92727    621\n",
       "Name: user_id, dtype: int64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_id_train.iloc[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "75220    1411\n",
       "48955     659\n",
       "44966     268\n",
       "13568     286\n",
       "92727     751\n",
       "Name: item_id, dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "item_id_train.iloc[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "75220    1\n",
       "48955    5\n",
       "44966    4\n",
       "13568    4\n",
       "92727    4\n",
       "Name: ratings, dtype: int64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rating_train.iloc[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. The model\n",
    "\n",
    "We can feed our dataset to the `FactorizationModel` class - a sklearn-like object that allows us to train and evaluate the explicit factorization models.\n",
    "\n",
    "Internally, the model uses the `Model_dot`(class to represents users and items. It's composed of a 4 `embedding` layers:\n",
    "\n",
    "- a `(num_users x latent_dim)` embedding layer to represent users,\n",
    "- a `(num_items x latent_dim)` embedding layer to represent items,\n",
    "- a `(num_users x 1)` embedding layer to represent user biases, and\n",
    "- a `(num_items x 1)` embedding layer to represent item biases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "#from torch.autograd import Variable\n",
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's generate [Embeddings](http://pytorch.org/docs/master/nn.html#embedding) for the users, _i.e._ a fixed-sized vector describing the user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[-0.9320, -0.6555, -1.0655],\n",
       "         [ 0.4524,  1.0263,  0.2686],\n",
       "         [-1.0496, -0.7165, -0.6973],\n",
       "         [-0.5366,  0.5336,  0.2259]],\n",
       "\n",
       "        [[-1.0496, -0.7165, -0.6973],\n",
       "         [-0.1849,  2.1344, -0.1380],\n",
       "         [ 0.4524,  1.0263,  0.2686],\n",
       "         [-1.0058,  0.2656,  1.5274]]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embedding_dim = 3\n",
    "embedding_user = nn.Embedding(total_user_id, embedding_dim)\n",
    "input = torch.LongTensor([[1,2,4,5],[4,3,2,0]])\n",
    "embedding_user(input)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure to check out ```torch_utils.py``` file to find the helper functions used in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import imp\n",
    "import torch_utils; imp.reload(torch_utils)\n",
    "\n",
    "from torch_utils import ScaledEmbedding, ZeroEmbedding\n",
    "\n",
    "class DotModel(nn.Module):\n",
    "    \n",
    "    def __init__(self,\n",
    "                 num_users,\n",
    "                 num_items,\n",
    "                 embedding_dim=32):\n",
    "        \n",
    "        super(DotModel, self).__init__()\n",
    "        \n",
    "        self.embedding_dim = embedding_dim\n",
    "        \n",
    "        self.user_embeddings = ScaledEmbedding(num_users, embedding_dim)\n",
    "        self.item_embeddings = ScaledEmbedding(num_items, embedding_dim)\n",
    "        self.user_biases = ZeroEmbedding(num_users, 1)\n",
    "        self.item_biases = ZeroEmbedding(num_items, 1)\n",
    "                \n",
    "        \n",
    "    def forward(self, user_ids, item_ids):\n",
    "        \n",
    "        user_embedding = self.user_embeddings(user_ids)\n",
    "        item_embedding = self.item_embeddings(item_ids)\n",
    "\n",
    "        user_embedding = user_embedding.squeeze()\n",
    "        item_embedding = item_embedding.squeeze()\n",
    "\n",
    "        user_bias = self.user_biases(user_ids).squeeze()\n",
    "        item_bias = self.item_biases(item_ids).squeeze()\n",
    "\n",
    "        dot = (user_embedding * item_embedding).sum(1)\n",
    "\n",
    "        return dot + user_bias + item_bias\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import imp\n",
    "import numpy as np\n",
    "\n",
    "import torch.optim as optim\n",
    "\n",
    "import torch_utils; imp.reload(torch_utils)\n",
    "from torch_utils import gpu, minibatch, shuffle, regression_loss\n",
    "\n",
    "class FactorizationModel(object):\n",
    "    \n",
    "    def __init__(self,\n",
    "                 embedding_dim=32,\n",
    "                 n_iter=10,\n",
    "                 batch_size=256,\n",
    "                 l2=0.0,\n",
    "                 learning_rate=1e-2,\n",
    "                 use_cuda=False,\n",
    "                 net=None,\n",
    "                 num_users=None,\n",
    "                 num_items=None, \n",
    "                 random_state=None):\n",
    "        \n",
    "        self._embedding_dim = embedding_dim\n",
    "        self._n_iter = n_iter\n",
    "        self._learning_rate = learning_rate\n",
    "        self._batch_size = batch_size\n",
    "        self._l2 = l2\n",
    "        self._use_cuda = use_cuda\n",
    "        \n",
    "        self._num_users = num_users\n",
    "        self._num_items = num_items\n",
    "        self._net = net\n",
    "        self._optimizer = None\n",
    "        self._loss_func = None\n",
    "        self._random_state = random_state or np.random.RandomState()\n",
    "             \n",
    "        \n",
    "    def _initialize(self):\n",
    "        if self._net is None:\n",
    "            self._net = gpu(DotModel(self._num_users, self._num_items, self._embedding_dim),self._use_cuda)\n",
    "        \n",
    "        self._optimizer = optim.Adam(\n",
    "                self._net.parameters(),\n",
    "                lr=self._learning_rate,\n",
    "                weight_decay=self._l2\n",
    "            )\n",
    "        \n",
    "        self._loss_func = regression_loss\n",
    "    \n",
    "    @property\n",
    "    def _initialized(self):\n",
    "        return self._optimizer is not None\n",
    "    \n",
    "    def __repr__(self):\n",
    "        return _repr_model(self)\n",
    "    \n",
    "    def fit(self, user_ids, item_ids, ratings, verbose=True):\n",
    "        \n",
    "        user_ids = user_ids.astype(np.int64)\n",
    "        item_ids = item_ids.astype(np.int64)\n",
    "        \n",
    "        if not self._initialized:\n",
    "            self._initialize()\n",
    "            \n",
    "        for epoch_num in range(self._n_iter):\n",
    "            users, items, ratingss = shuffle(user_ids,\n",
    "                                            item_ids,\n",
    "                                            ratings)\n",
    "\n",
    "            user_ids_tensor = gpu(torch.from_numpy(users),\n",
    "                                  self._use_cuda)\n",
    "            item_ids_tensor = gpu(torch.from_numpy(items),\n",
    "                                  self._use_cuda)\n",
    "            ratings_tensor = gpu(torch.from_numpy(ratingss),\n",
    "                                 self._use_cuda)\n",
    "            epoch_loss = 0.0\n",
    "\n",
    "            for (minibatch_num,\n",
    "                 (batch_user,\n",
    "                  batch_item,\n",
    "                  batch_rating)) in enumerate(minibatch(self._batch_size,\n",
    "                                                         user_ids_tensor,\n",
    "                                                         item_ids_tensor,\n",
    "                                                         ratings_tensor)):\n",
    "                \n",
    "                \n",
    "        \n",
    "                predictions = self._net(batch_user, batch_item)\n",
    "\n",
    "                self._optimizer.zero_grad()\n",
    "                \n",
    "                loss = self._loss_func(batch_rating, predictions)\n",
    "                \n",
    "                epoch_loss = epoch_loss + loss.data.item()\n",
    "                \n",
    "                loss.backward()\n",
    "                self._optimizer.step()\n",
    "                \n",
    "            \n",
    "            epoch_loss = epoch_loss / (minibatch_num + 1)\n",
    "\n",
    "            if verbose:\n",
    "                print('Epoch {}: loss {}'.format(epoch_num, epoch_loss))\n",
    "        \n",
    "            if np.isnan(epoch_loss) or epoch_loss == 0.0:\n",
    "                raise ValueError('Degenerate epoch loss: {}'\n",
    "                                 .format(epoch_loss))\n",
    "    \n",
    "    \n",
    "    def test(self,user_ids, item_ids, ratings):\n",
    "        self._net.train(False)\n",
    "        user_ids = user_ids.astype(np.int64)\n",
    "        item_ids = item_ids.astype(np.int64)\n",
    "        \n",
    "        user_ids_tensor = gpu(torch.from_numpy(user_ids),\n",
    "                                  self._use_cuda)\n",
    "        item_ids_tensor = gpu(torch.from_numpy(item_ids),\n",
    "                                  self._use_cuda)\n",
    "        ratings_tensor = gpu(torch.from_numpy(ratings),\n",
    "                                 self._use_cuda)\n",
    "               \n",
    "        predictions = self._net(user_ids_tensor, item_ids_tensor)\n",
    "        \n",
    "        loss = self._loss_func(ratings_tensor, predictions)\n",
    "        return loss.data.item()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = FactorizationModel(embedding_dim=128,  # latent dimensionality\n",
    "                                   n_iter=10,  # number of epochs of training\n",
    "                                   batch_size=1024,  # minibatch size\n",
    "                                   learning_rate=1e-3,\n",
    "                                   l2=1e-9,  # strength of L2 regularization\n",
    "                                   use_cuda=torch.cuda.is_available(),\n",
    "                                   num_users=total_user_id+1,\n",
    "                                   num_items=total_item_id+1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "user_ids_train_np = user_id_train.as_matrix().astype(np.int32)\n",
    "item_ids_train_np = item_id_train.as_matrix().astype(np.int32)\n",
    "ratings_train_np = rating_train.as_matrix().astype(np.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0: loss 13.133116082300115\n",
      "Epoch 1: loss 7.496939776818963\n",
      "Epoch 2: loss 1.808433080021339\n",
      "Epoch 3: loss 1.0783175373379188\n",
      "Epoch 4: loss 0.9451855775676196\n",
      "Epoch 5: loss 0.8943392981456805\n",
      "Epoch 6: loss 0.8713056995898862\n",
      "Epoch 7: loss 0.8526222585122797\n",
      "Epoch 8: loss 0.8382889749128607\n",
      "Epoch 9: loss 0.8239792834354353\n"
     ]
    }
   ],
   "source": [
    "model.fit(user_ids_train_np, item_ids_train_np, ratings_train_np)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DotModel(\n",
      "  (user_embeddings): ScaledEmbedding(944, 128)\n",
      "  (item_embeddings): ScaledEmbedding(1683, 128)\n",
      "  (user_biases): ZeroEmbedding(944, 1)\n",
      "  (item_biases): ZeroEmbedding(1683, 1)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "print(model._net)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8865973353385925"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_ids_test_np = user_id_test.as_matrix().astype(np.int64)\n",
    "item_ids_test_np = item_id_test.as_matrix().astype(np.int64)\n",
    "ratings_test_np = rating_test.as_matrix().astype(np.float32)\n",
    "model.test(user_ids_test_np, item_ids_test_np, ratings_test_np  )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks like we are already overfitting..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Analysing and interpreting the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "user_emb_np = model._net.user_embeddings.weight.data.numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "item_emb_np = model._net.item_embeddings.weight.data.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[How to Use t-SNE Effectively](https://distill.pub/2016/misread-tsne/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.manifold import TSNE\n",
    "\n",
    "item_tsne = TSNE(perplexity=30).fit_transform(item_emb_np)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAI1CAYAAADVQv5HAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsvd+PE9e69/mU3QXYbAl3zuFixxOSzLkACXFCH3hPopcruAjSyxB5wt4bZbL/iPBGLfXWoNBsMQpS60zyR+wo6iTsscjwSuQiXHFEzsDr7oOQQK/2yQ7IOdIwE5x3Qhtwu2suzDLLVetZP+qHXWV/PzcJbrtcrlq11nc9P70gCAgAAAAAYFYpTfoEAAAAAAAmCcQQAAAAAGYaiCEAAAAAzDQQQwAAAACYaSCGAAAAADDTQAwBAAAAYKaBGAIAAADATAMxBAAAAICZBmIIAAAAADMNxBAAAAAAZpo5lzf/7d/+bfDaa69ldCoAAAAAAOlx69at/ycIgt2m9zmJoddee41u3rwZ/6wAAAAAAMaE53k/2LwPbjIAAAAAzDQQQwAAAACYaSCGAAAAADDTQAwBAAAAYKaBGAIAAADATAMxBAAAAICZBmIIAAAAADMNxBAAAAAAZhqIIQAAAADMNBBDAAAAAJhpIIYAAAAAMNNADAEAAABgpoEYAgAAAMBMAzEEAAAAgJkGYggAAAAAMw3EEAAAAABmGoghAAAAAMw0EEMAAAAAmGkghgAAAAAw08xN+gQAAACAPNFstWnl6j36sdOll2sVWjy+lxoL9UmfFsgQLwgC6zcfPnw4uHnzZoanAwAAAEyOZqtNf/jzber2+sPXPCIKiKgOYVQ4PM+7FQTBYeP7IIYAAACAgRD68It16mvWRQijYmErhhAzBAAAYOYRFiGdECIaCCEionanS3/4821qttrZnxzIHIghAAAAM8/K1XsjrjEbur0+rVy9l9EZgXECMQQAAGCmabba1O50Y3223enCOjQFQAwBAACYWYR7LAlwlxUfiCEAAAAzSxz3WBi4y4oPxBAAAICZJa57LMyPKR0HTAaIIQAAADNJs9UmL6VjvVyrpHQkMAkghgAAAMwkK1fvkX2lPT1H9+1O6UhgEkAMAQAAmEnSdG1dutVGEHWBgRgCAAAwk+yq+KkdC0HUxQZiCAAAwEzipRUw9BwEURcXiCEAAAAzSWejl+rxEERdXCCGAAAAzCRpihe/7NHi8b2pHQ+MF4ghAAAAM8ni8b2ppdbv3DaHDvYFBmIIAADATNJYqKeWWv9zN12XGxgvEEMAAABmlnpKrjLECxUbiCEAAAAzSxqusopfRrxQwYEYAgAAMLMkdZXNV336+N0DiBcqOHOTPgEAAABgktRrldgNW1sfvZ3y2YBJAMsQAACAmWbx+F6q+GXnz6UVbwQmDyxDAAAAZhrh4lq5eo/anS55REbXGeKEpguIIQAAADNPY6E+FEXNVpuWL9+hTihdXoikeq1Ci8f3Ik5oioAYAgAAACSEMGq22rRy9R792OnSyxBAUw3EEAAAAKBAthaB6QYB1AAAAACYaSCGAAAAADDTQAwBAAAAYKaBGAIAAADATAMxBAAAAICZBmIIAAAAADMNxBAAAAAAZhqIIQAAAADMNBBDAAAAAJhpIIYAAAAAMNNADAEAAABgpoEYAgAAAMBMAzEEAAAAgJkGYggAAAAAMw3EEAAAAABmGoghAAAAAMw0EEMAAAAAmGkghgAAAAAw00AMAQAAAGCmgRgCAAAAwEwDMQQAAACAmQZiCAAAAAAzDcQQAAAAAGYaiCEAAAAAzDQQQwAAAACYaSCGAAAAADDTQAwBAAAAYKaBGAIAAADATAMxBAAAAICZBmIIAAAAADMNxBAAAAAAZhqIIQAAAADMNBBDAAAAAJhpIIYAAAAAMNNADAEAAABgpoEYAgAAAMBMAzEEAAAAgJkGYggAAAAAMw3EEAAAAABmGoghAAAAAMw0EEMAAAAAmGkghgAAAAAw00AMAQAAAGCmmZv0CQAAwKRottq0cvUe/djp0su1Ci0e30uNhfqkTwsAMGYghgAAM0mz1aY//Pk2dXt9IiJqd7r0hz/fJiKCIAJgxoAYAgBMJc1Wm5Yv36FOt0dERPNVn86d3D8UOitX7w2FkKDb69PK1XsQQwDMGBBDAICJk7a7qtlq0+KX69TbCoavPdro0eJX60Q0sPz82OkqP8u9DgCYXhBADQCYKMJd1e50KaAX7qpmqx37mCtX740IIUGvH9DK1XtERPRyraL87K6KT81Wm45c/JZeX7pCRy5+m+hcAAD5B2IIADBRdO6quOisO+Jvi8f3kl/yIn///55u0n9eXRsRZ4tfrkMQATDFQAwBACZKHHeVyXLDWX3kvzUW6vSrHdFIgf5WQFuh13pbAS1fvsMeEwBQbCCGAAAThRMu3Os2brWj+3YrP+uXPVo8vnf4785Gz/o8RSA2AGD6gBgCAEyUxeN7qeKXR16r+OUR0SJjcqs1W226dEvt0hIxQ0I46SxIAIDZAWIIADBRGgt1+vjdA1SvVcgjonqtQh+/e4DNJjO51VRiSUa2JKmEGMd81bd6HwCgeCC1HgAwcRoLdetU+pdrFWorBJGw8tikxgtL0vWlY0REw7T+kudRP4hmoRENUvOPXPwWVaoBmEIghgAAuUeuQ1Sr+uSXvJHUedmtxomlMEI0yULs9aUr2s+gSjUA0wncZACAXBMOmH600RsRQvNVf8StZuv6erlWiWSl7aqYXWFJ0/4BAPkDliEAQG5pttr04RfrrOuKiOhJbzQRXm63ISxJvzzZjFiSju7bHelN5pe9iNVJBapUAzBdQAwBACaOqh0HEdEf/nxbK4SI1P3EwjFI4T5lO/wSXfnXf48EWvf6wTBQ+pEm7R5ZaABMFxBDAICJwnWP3+GXtFlhMjaWmqebLyxIOqHT2ejRy7UK+x5d2j8AoJhADAEAJgpXN8hWCBGZLTWmdPvwsXTiSpf2DwAoJgigBgBMlKTxNzaWGtvvEMfixFW9VoEQAmAKgWUIAKBEFceThRDgUuFrFZ+ebm6NWHQ8IgqIqPy8HlDd8rx06fbzVX/oGpOPJbvuiOAeA2CagRgCAETg4niI0q+vs3h8r1J4LL+zn4hoJPC5VvXp3Mn9xnMIC7nX/oYXQ/+9uxl5LZyRJgd1H7n4beYCEQAwXrzAkKkhc/jw4eDmzZsZng4AIA8cufitUjyUPY+2giB1IcBZocKijGgglFRxO+IY7U53aEFyhTu2OH74XMT32FqoAADjxfO8W0EQHDa9D5YhAEAEzooi0tzTshSFRdAnpw+OHI8Lrl6+fGfkc0f37aZLt9rD98YRQuLY4TR93bmI70FlagCKDcQQACBCWdOjSyBXYg4Lk2t3HxpdSWebt+mzG/eVgkIckxNlnW5v6Dprd7ojx0mKqREsh05IAQDyDcQQACCCSQgJhICRY4v+dON+5O9EFCmCqBIw3V6fzn99h570tpxS69MSQkR8mr5NzzNUpgagmCC1HgAQoW5ZYbnseUbRourltXL1HitgHm30nIRQXMqlQesNGY+Iju7brXy/Tc8zm95mAID8ATEEAIhg2+zU1oIUtpjEtaDUa5Vhu4wwnvJVnv5WQH7ZG/lcQESXbrWp2WpH3t9YqNPH7x7QCsXHzzaVnwUA5BuIIQBABJuFv1bxrS1IYdcT54rynh9XRb1WoetLx+jcyf0RoVbxy/T+W3vYz3Js9LaUrjquK31joU7Xl47RXy+eUIqyXj9AR3sACgjEEABAiVj4Pz19MCI+PBoEMT9+ukl+2WyTCRcrVFmePCJ6/609tPyOWuyIY8hCzaOBSPr43QN0oXGAdm5PJwzSxnLVYXqXIW4IgOKBAGoAgBYR+Hz+6zvD5qXCmtLp9sgveTRf9dnGpvNVP5JhxRU1DKfVc38Ld6UXuAgRXS0im670XEA1OtoDUDwghgAAWpqt9ogQCtPbCpRVnAXPNreo2WorBRGXhq77mw5TxpcQQDohpGq7oSoKyVXORssOAIoHKlADAFiarTYtfrlOva1kyetcpea0+5+pqkSH0dVQUlWS1lXBJtJbsAAAkwUVqAEAiVm+fCexECJSV2omIm3/M51Q4v7WWKjTzR9+os+/e8AKHu51j4iuLx2LvM5VwV65eo+uLx2D+AFgCoAYAgCwiCrPaSJna3Eig4gXSqa/XbrV1qb8c5YhOdZHFlvckRAoDcD0ADEEAIhNySOKYzjSCYkfO12tNUb8v+3fZCp+mU4dqo/0MROvi1gfG1cbEVGNqXcEACgeSK0HALCwBQ49ok9PH6T//XfRtHsbXq5VtG0vdP3BuL+1O11t8LScgq9KzZcz3GwqYP/yBAUWAZgWYBkCALCcO7mfFr9ap17/hfnHL3u08ps3tGnw4rV2pxvJ3JKtMFw2FtekVQgoTvTossTazy1ORPpsNVv3V28rQGNWAKYEZJMBALQkzfiyCYRud7rDWJ56bdD5XuXKEhlcNm4sDlHc8ULjgPL8Np5tsmUEVMf6/uKJWOcBAMge22wyiCEAgDVpp8KLY6osRKcO1ena3YesiPpgdS32d3pE9Mnpg0QUFVZ+ySPyaMQaxlmcRIsQAEA+QWo9ACBVwqIlnAofFy5Y+trdh6zQaCzUWVea5xGZ9njB8+8V3yXT2wqoVvFp5/a5oRDjLFUosAjAdADLEADAiiMXv1WKj7B1xNV69PrSFTbOp/48mFp1HM6i5FFAG70t4+8RHdV0362rbXR0327WcgUAyAewDAEAUkWX4SWIYz3iWmh49CJQWnUcrr/ZGUv3mSkYO/ydctB1VlYyAMBkgGUIgBnFZMGxDSyWLUO21qPweYQtPElidLhzkHEJxp6v+lTdNjdynTgXHWKIAMgXsAwBAFhMlg3V3/2SR37ZGwksDsfN2FiPwqgsPJyY0R1HzkwLiym/7NHObXP0c7enFH6cuCEierTRG4pAcZ048RS2kqFvGQDFAGIIgBlEV+FZBCfbBBaHF3hOyIQLLKqEgmxR4aw7wfO/meKH5M70quarMsL9ZWNRIhpcJ1NLD7jRACgWqEANwAxisuBwf/+526PrS8fo+4snlE1KF4/vjVSkDluPhFBoP+/7JYSCXM158fhe8sseqVC9XyXehBCybaaqOneOfhBof6epnQgAIF9ADAEwg+haYdj8vdlq05GL39LrS1foyMVvh8KksVDXtrogchAKmnDG8PvjuOfCqM69VlG3I5mv+rTDfzF9VvwS7fBLdGZ1TWthQnNXAPIJ3GQAzCCLx/eyrTBMf2+22rT45Tr1nndobXe6tPjlOhFFs65U2AiXlav3hse3OY6te85E+NxVwd1+2aNfnmyOnF+3t0Xd5+n8qpiluOcDABgPsAwBMGOIeJ1ur0+e5ImSLR3CSiJbRsTfly/fiQiV3lZAy5fvWH2/yepEZGdBkd9v456zRbZ6rVy9R6cO1UesRTu3zRmFmohZSuN8AADZAzEEwAwhx+sQjVZqfrTRi8TiPN3civy901X37eJeD2MjXEwWlPD7bdxzNqjimS7datPi8b3DOKmfLX9nQETl52qz7Hl06pDeYgYAmBxwkwEwQ6jidWRMGWVxm6PKcMUSZaGgctOZssNM7jkbTFl2RLxLLoxHNMw46wcBXbrVpsOvvgRBBEAOgRgCYIawcT+J99gs+DI7t+kzsVzq7tgIpiywiWdSCbUwqpihsKgCAOQHiCEAZggbq4ZwUXG1dDj8Mu91j1N3Jw1Ljys2gdgqoSb3KdtV8VmXYbvTpSMXv0UhRgByBmKGAJghFo/vJb+krt9DNBqL4yKEiEgbS1OUuju2gdiNhTotHt9LLz9vJHvt7kNaPL6XPjl9cCTOKozot8bVVwIATAZYhgCYIs42b9Pn3z2gfhBQ2fPovTdfoQuNA6NvYrRQOBanxlg4TNWXVaRRBygLVK67j989YHTPcZauHX5J6z6D6wyAfAIxBMCUcLZ5m/504/7w3/0gGP5bCKKVq/dGeosJwg1Gm602PX62GXmfX/Lo9D++QpdutdkaRSrSqgOUJpyg+fjdA8Zmq5ylK06A+aQFIQAAbjIApobPv3tgfN3WQsOJpl/tmKMLjQPOaewq95NHREf37WY/kzVJXHdpCpiS50UqeQMAxgssQwBMCVyMj/y6rYWGW+wfbfTo9aUrzsG/jYU63fzhJ/rsxv2hqyggmki6udzdXoVtwUfXbDsOcX/anS59sLpGH6yuGZvLAgDSBZYhAKaEsqcOBpJftw0Q1rmvXIN/RUXnP0lCSDDuIOpw0UkVNq4726au9VqF5qvq/mY6EFwNwHiBGAJgSnjvzVeMr5sqNQvhIvpr6bARMjbio93pjs1NZCo6adsyQ76OHB4RXV86RudO7tdm8HHkMdsOgGkFbjIApgQRJG3KJuPq94QDikV/LV2CvcmlZBIfAtnaJM4xC3TnW6v4tPzO/tS+W1iYxPGWL98ZZufNV316tGFu64HgagDGA8QQAFPEhcaBaCq9JSrhItpfEKkrUovgXy6GyHUxzzrVXBfro6sPFOZs8/ZI/FMYv+xFeqeFf9PCH78xCiJ0uQdgPMBNBgAgIr79xo+dLhsj0w8CbQxRnMU8S2uILnvN1i3VbLW1QoiI9Oa055w7uZ/8sl0BTABAtkAMAQCo2WqzMUIv1yqRWCNVsLZKTHDi4/dv7WHjbQKizOKHrt19qP27SoiJOCoR17R8+Y5R6/S2AqOwaizUaeU3bwyv6XzVp1rFty5XAABID7jJAAC0cvWecoH3iIbWCdnV8/rSFeVxwmKCEx+ifQXX8NQlfsilAazJ6hS2ZKkKM9rS7nSp2Wprz1/lPhO/58zqGq1cvYcUewDGACxDAABWJAQ0KkaElYSzjJQ8b8SioyvyaMrIcs1Ws0n517ntVG4p2wBwDtf0eNffAwBIB4ghAAArEmShYpMm3w+CkcWbO66caXV96RjroouTraYTUVzs03zVV7qlksYvuabHF6WhLQDTBsQQAMCqGKOtlURevJMWeTTFD7k2gFXVWfr09EFqffS20hXFndd81R85xu/f2qN8n+5cXN6LFHsAsgUxQwCAoRDQxd7EWdRtjktExvihxa/WafnyHfq52xs5hq69CBdLxNVZUqE6r4pfpnMno/WIrt19mLgZbR4b2gIwC0AMAQCIiC/GKHDpxyUv3jbiQxZNqu/o9YNhwUI5uFolVjwieu1vKsqO9PJ32aATc2GxdXTfbrp0qx0RTi7p8Zz4Qoo9ANniBUxzRxWHDx8Obt68meHpAADySjizimhQXJCCQSq5oOKXE6WFv750xaZMD9VrFbq+dExZAJGrnC0+Y4MuS011LSp+mU4dqtO1uw/px06XdlV88jyizkbPqbGtS3YcAECP53m3giA4bHofLEMAACs4K4nqtXAGmsvibmuBEq64a3cfRoQPJ6ZsXX2qlHrZssQFOl+7+5CuLx0zfl6HixsPAJAOEEMAAGu4hZpbvOOIAl38kIxwxbnEMpnahwh0WV2Nhbox0Nn0eQGsQADkA2STAQAyI06qeGOhTqcO1dl0e6LROBouuFj1eVP7EIFJ7JhKBthkhaGmEAD5AWIIAJAZcVPFVa4vQbhVBZe+//7zlh8eqYWRTpSZxI6pZIDp80SoKQRAnoAYAgBkho0oUMGJJY+Iri8dG3ElqWoHffzuAbrQOEDXl47RJ6cPOscQcT3VxOvcd5oEmpwVhppCAOQHxAwBAGIjx7zUqj4FAY3UAoqbKu5Sb4eLuxGv64KxOVGm66km0PUVE9dj+1wpUhvJ9BtFSxPEDgEwPmAZAgDEIhzz8mijR51ubyT+hYi0FhQO28rVXNzN2eZtY+sQ8T0q4lhtVNfj6eYWfXL6YMSaxf1GomhLEwBA9sAyBACIhak9h4h/UQkBE7aVq7m4m8+/e0B9Qw21+arPnlecStC2GWQC8dqHX6xHzlX3OQBA+sAyBACIhU1sS5L4F9HE9fuLJ1hBxR3fJIRESw0OldXGo4HlieuVFsea1Fio0xZzrogdAmB8QAwBAGJh0y8r655a3PHLHp+Yb+OqkwOkiUYrWnMp8HGbzcYNMgcApAfEEAAgFlzMi2AcPbW42KL33nxF+fqnTPyOCmGZqtcqkWw0VQq87nroagjZxkcBALIDMUMAgFiE43rkbDLx/2dW1+j813ciWWZpxcLoYosOv/qSdXVnXSVoW/eXqdksFwdkGx8FAMgONGoFAKSKqompTNJGrq7nYhIZXNNVcY5HLn6rFDf158dTHZ9rNusR0fcXT6T8KwEAHLaNWuEmAwCkim2WWdbYtrswVYLm3FhH9+1mj484IACKBcQQACBVss4ys8W23YXJDcZVm75296Hy+MuX7yAOCICCgZghAECqcDV6wu/JGttYH5uaQqpq02dW15TH73R7RDQoNok4IACKAcQQACBVFo/vpTOra2w/sHFZSGwLJ8ZtGVKr+vRoo6f8W9xik3nEJu4KgKIDNxkAIFUaC3V6/609yk7x81V/bMHTtq4qU9NVFc1Wm355ssn+vd3p0utLV7T1hYqAbdwVAEUH2WQAgEzIg0Uhq3PgMsxUjDN7Lm10mXTXl45N4IwAcMM2mwxuMgBAJqjibKblHFwCwIvcZyxOixGZPAhiAGyAmwwAABxxDQAvap8x7neWPM/oKoOLDRQJiCEAAHCEi0ear/rK9xe1vhDXYqQfBEZhY1vaAIA8ADEEAACOcEHXJ/7+15HA8SLXFxK/U9X41iRsOGuYbawVAOMEYggAAGIgGrl+f/HEMJj40q32SEkBj4hOHZp87FQSGgt12mISbXTuP84a5hHBVQZyB8QQAACkgMotFBDRtbsPJ3NCKRKnvcji8b3K8goBEVxlIHcgmwwAAFIgaeZVnrEtTBnOHuMKt0zDNQHTBSxDAACQAtPcnLWxUKdTh+rD2KGy50Xcf6rsMZVliGg6rgmYLiCGAAAgBaa5OWuz1aZLt9rUfx471A8CunSrPRL7s3z5jtJNGGZargmYLiCGAAAgBeK09SgKpjT5Zqs9bFBrougB5WA6QcwQAACkRB6qbmeBKR5q+fId62NNQ0A5mD5gGQIAAKBFV4n69aUr1lYhIgRPg3wCMQQAAECLrhK1favvAQENGsCi1hDIE3CTAQAA0CJcfyJtvuR5w2DqOIg+ZfKxAZgksAwBAACwYuPZJgVEiYSQAH3KQJ6AZQgAAICWZqtNi1+tU6+fXATJIH4I5AVYhgAAAGhZuXovdSFEhOKLID9ADAEAANAS14Kzc1uZahWfiChSjRrFF0GegJsMxOJs8zZ9/t0D6gcBlT2P3nvzFbrQODDp0wITJNyXavH4XgTHTgm1qk+PNvTp8x4NLD3c/cf4AHkGYgg4c7Z5m/504/7w3/0gGP4bgmg2EX2pRJViZAtND81Wm355sml8X0CkFTrTWpASTAdwkwFnPv/ugfL1zySBBGYLU7sGUFxWrt6j3pZdvJBo0PqHP98e1hFqttp05OK39PrSFdQXArkFliHgDJdWG9Bg4rPd/cFsPj2Y2jWA4hLnHspCGBZDUARgGQJOmHZ1tpYA4VZpd7rK3SQoFlxWELKFig93D8ueFwmKlml3uvThF+uwGIJCADEErBECRoftLhJulelC1a4B2ULTAXdv/+l3b9D3F09QnRFLHvFWZFgMQd6AGALWqARMGFtLANwq00VjoU4fv3uA6rUKeURUr1Xo43cPwBUyBZjuLde3TBdlBIshyBuIGQLWmISKiyVgV8VXdrre9bwmCSgeyBaaXnT3trFQp5s//DSSYarDI4LFEOQOiCFgRAQ663Z681Wfzp3cb70YekywAfc6yDeqYHgiQoD8jHDlX//d+r0BIXga5A+IIaAlXD+G40lvy+m4HaaAG/c6yC+qGkOLX60TBTRMyUYWUT5JK6PTVJBRpgbrL8ghiBkCWmzihIjcg59ts49QoyT/qMZIrx9EatMgQD5fqDI6P1hdo4U/fpPpcwbrL8gjEENAi0tAs8t7j+7bHUnL9WgwIQvRc7Z5m86sriH9PudkNUZAtnAbnUcbPefnzMXaA+svyCMQQ0CLS9aH7XubrTZdutWOxCCJfws3y59u3I+8R7YuwGo0eZqtNpUctvrIIsoPOmHqasVbfmc/+SW7cYAxAPIIxBDQokqb9Use+eXRic8lk8zG9dbr8+HaP3a6KNqYA8Q9UNWS8cteZHH0Sx5tPNuEeM0JJlHiYsVrLNRp5bdvGC1EqD0F8grEENASrjFSq/j0qx1z1OsPutUTudeUSeoqeblWQdHGHMCJ2rLn0cpv3ogsjr2tgB5t9CBecwJXH0jgasFpLNRp7dzb9Onpg8P5Yr7qU63io/YUyD3IJgNGRI2RcNZQPwiGOz2XCe7lWoXaMQWRqFFyZnVN+XfEpIwP7lr3g4BWrt6jdqerbdfQ7fXpwy/WiQgZZpNAXPPly3ciNb+SWHBQbwoUEViGgDVpWWNMO1JBeHB6RPT+W3uosVBHL6wxoYvL4q61CIQn0lchJhoIJ1iIJofKmgMLDphFYBkC1qTVQkNMsnJ9k9f+pkL//JefRhZPuXJRuKjj4vG9kfpHiEdIj2arHbEYhGsFqe4BkVkAhen2+rR8+Y528W222nT+6zvDeja1ik/L79gX+QR6YM0Bs44XMI30VBw+fDi4efNmhqcD8syRi98q3Vv158XahLjZVfHJ8wYptLaF3LhjCzwaLLL1WoWO7ttN1+4+pHanS2XPo34QDM+BCFWPk2IqtFmvVej60rHhe2WXmKsQkvn09EHlvWq22rT41XokqN4vebTy2zdwfwEALJ7n3QqC4LDxfRBDwBbVIlnxy3TqUJ0u3Wqzi2fFLxvN7q8vXYm9kIrjE5Hy/GDyd8NGmH5/8YTTZ2yQRZbtsWsVn3Zun4P4BQAosRVDiBkC1nDdq6/dfahNlbeJK0oS6yOOjwyzdDC5PVX3Ko3A9Thu2E63h/IKAIDEQAwBJxoLdbq+dIy+v3hi6BqzsQiYFsuksT4/drqpxTTNOjphysVlpRG4nkZQPMQvACAOEEMgFnLRQxtMC1pjoU7z1fgNHF+uVZBhlhJctt981WeFhDXKAAAgAElEQVRdjqrPVPwy/f6tPdGinYqCjLrg98XjeyNFPnVA/AIAXIEYArGwbeAqOLpvt/E9507ut0q5DyMWUm5BRoaZGyp36KenD1Lro7fZeBzOhXqhcSDy+un/8Ar9aseLRNZahRdZ4tgrv3nDWixD/AIAXEFqPYiF6+772t2HkddEJpIc/PrxuwdGUqhNlD0vspCGj0k0CMJFkK09cVKtuc/Ir6uC8J9ubkU+wx3DFKgN8QsAiAMsQyAWrrvvsHjieosREbU+ihaB4wj3xZJjmkRmUvh7Plhdo4Pnv0GgLUOWDXCTBrnrCnaiWCAAIC5IrZ9yVNaXNBYLLs1+h19SWnXCadO6mkWu6dW69Pm4n5tVuPtqc510Y02uR8ThEVmN0bPN2/TZjfsjpRhwLwEAKpBaDzLt7M7FiKjiflSuC87N1u50lRaJo/t2s32udJYFnTsPmUdR4lpudGPNNtjedoxeu/swUpMK9xKA7MjSWpwXEDM0xegWtjR20HID15Wr9+jM6hq9XKvQqUN1unb3odYapWvWGnabERFdutXWFmXkRI+pKSwyj0aJW57AJKJcgu27vT59sLpGK1fvKccOdz+FkEZcGADpEbYWh9vyTAsQQ1NM1nV3uP5Vl261jS4Lrq+VTLfXp/Nf36HqtjnjYsrFMJm+B5lHo3DiUXedmq02K1CSjDXVpNtstbVtP8JCepomawAmAbfR+fCLdSKanmcMbrKCEMdMmWXdHbFbkIWQwMZlEXazcTza6BndK7oMIvE9qrRsZB5FcS1PIMYBR8nzEvUrC4+llav3rI4HtxkA6cBtaPpBQB+srtHCH6PJKEV0q8EyVABUZsrFL9fp/Nd3tM1Qs+zsbqozZGMRkFOudYHOohmrirqFSyTszkOKPY+4HrbXyTQOuPvmgjyWXCxNcIECkBxTqMGjjd6IJbaobjVkkxUAmyaYXDYN11V8vurTuZP7Yw9OU2NVLiuMo9lq0wera+zfK34ZDVhzSJIGu7bIY8mlIazrGAQARFFlmKoQzxv3jHreoMCqbgOfBcgmmyJsdricW6CxUB+0MyiNuisebfRo8av12ObLOP2rdDQW6lSrqCsMi0y1cOYahNDkSSvmqlbxrVyZKjeea3sPAIA9ItSg7Olb4oh1iluvgmCw7uS1qTLcZDEYt7vFZKYUcNk0K1fvUW8run/v9YPYmWVcYHISi9PyO/tZt16cisggW5qtNm0823T6zHzVpye9rZF77Jc8evxsk3r90TFaq/i0/M7oWOLceKrXMF4ASAfxLNkko9iuV2lmNqcBxJAjk/CH2mReCVTZNDrLUty4CtfYkkkdE2SDrelcpuKX6dzJ/UQ0eo83nm0qC3Xu3D5nbO8Rfh0AkA3i+QpnEBMNnu2j+3YPXWS6jE+ZPMX1IWbIEdfKyWnRbLWdenbJ56SLs0BcBYiDS+wO0cDK43mkjBfQxR3VaxUrYYzgeADGR/h5O7pvN1261XbaHBEN5oW1c29ndJYDEDOUEVnX7uFoLNSpus3NkCfOScQMhfHLHuIqgDVyuqyLEKr6JXq6ucXGC3BxR97z95piDLKstA4AiCL3gFw8vpc+/+6BsxAiInr8bDM3zyncZI7EKUoXB9VO11VwiXNSmTeTZpOB2SKOW0zQ3dyisAFaDvhXxR2pzOxcjEHWldaLBCxkYJyIeSFuCY0kcatpAzHkSJa1ewRcXNKuiq8scqgifE4IQAauyAtrSVPryQT3MTGuw0Kmphnnqo3IpKy1eUM1b5xZXaObP/xEFxoHJnx2YBox1RmzIS/PKcSQI2kH+ap2ctxOd4dfsgpM84jo1KHiiB/sZvNHeGFNUjyRK5pZ9jzlRLpz+xzt3D7HuuJeW7oyUmxzXNbavKOaNwIi+uzGfTr86kt4pkDqpCFkaoqSGpMAMUMxEP7ST04fJCKiM6trsUqOc7EO3CLQ2ejR+2/tMR43oEFn7yKAeI98ksaOj2hgoXzvzVeULT44gfVjp0uLx/dq27TI48S1hci0wtZ3IUJrEpAJaWw4UihSnwoQQzFJYxHnLEBccauXaxW60DhAv39rj3ahIMqP6dGEqds5mAxpjB9RHPNC44CyaGZd0zuvsVA3WkDluKBTh+rD56bseYWyjKaFbmEqynwAisXRfbsTH8M29CNr4CaLCbeIf7C6RitX71m5enQN8FTtJ8RO90LjAB1+9SVlvQdBUVwEiPfIJ7aF01SoWqVwMWu6+Lu6xTn82OlSs9WmS7faQ0tTPwjo0q32zLmGFo/vZVvaFGU+AMUiDQ+ERwPjwqSfVViGYqJbrG2tRNwENV/1rdpPPN3cUn6+SC4C7hpg8p4snOvp92/tibxe8gZ9h4j0VplwJ2si0o5z1TmEKXkenf/6jnJj8uEX64Xqmp0GZVUJjRJKaIBsSGPTmhc3LixDDKagXtPO2Sa1d/H4Xlr8aj3ShuCXJ4NUY10xRC6mo+x5herbNY7sPOCOLlHg8KsvDV/fVfFHWmlwVhkuQ/Ljdw+w41w+B+5Z6wcBW4hUWIranS4tfrVOy5fv0M/d8TaJHCcrV+9RX9F251c71JW8AUhKEguyTB48AbAMKbCJB7LZtZpucGOhTnOKnVxvKzAqZe7YW0FQqIlPNAEsShPWsHVjmi0OcmG160vHhvdEfn3n9rmImFfFfMWNDRONhk3PmoleP6BON79NItOAmxM6DlXrAXAhjWeTKB+eAFiGFNgUcbPZtZpucLPVpm5P7eoyCalpSicuSg2kSfSlyzu2MV/cM2Kzq0wrs01mGosyTtOcAIqByoLsainyiHLhCYBlSAE3wbefB2sKxA7509MHY6X26nbFpgkM6cTjB5lvUWxjvrgMSe51QbPVtp5cxZFMxxTkwTSfJpgTwCQIW5C5LFGOgPKxmYQYUqATIirzelxXj24yNk1gWbuXZskdZItOJM/qNbJdgLmaQrpijsISZ0tAg+fgn373hpXpftosJkVzOYPpxNV15iqesgJuMgWqoF6Byrwet4IyZ1Kcr/pWx8/KvZSGO2gaq0rrTMCz6jJrLNTp5g8/0effPaB+ELDZZFyavG4ijOMea3e6yl58YabVYpLlnDBtzzPIhrDrrFb16Zcnm9RTBPfn6TmEZUiB2GFxyBaCJMUXuV31uZP7Uzm+K8Ia9MHqWiJ30LRWlTbteGbRZcbV+Anfa1sLkmyR1LnHOEeYcJE1Fuq0c7t6r1e0jMtJo3qez6yu0dmmvdUOzBay66z10du08ts3hhsf8YzmzXIJyxBDY6HOBkfL5vUkHbNt+pyZjp/Wjs2mK7mImVJZrdqd7rAHlaoX1TQErNoEzU9bHIoJ2/FvM9ZtxiCRvhijPO6mJeNy0qDnGUhKEZJkIIaeoxIVXA2co/t205GL39KPz3dKKmwXRdMg0WXrpJndZOuSkI/PNfPU9ZwqOuJ+Hbn47VRm7qieAyJexKRZQdxmDAprEidIZbcbsqvSwdTzLO+LHAA2zKybTDbHHzz/DS1+tR5x6xBFK+SeOlSnS7faw/dy7Kqk04lXl62TZnaT7eIlH981piPPi5BrwPg0Zu6o3CGLX63T4pfRZ0NcH+6ehse/javFNAZls7rN9Z/GezQJ0PMMzAIzKYbCE3On22MLx4XTBq/dfWglACyze41wcSobzzZTddW4CBVxfJfvyfMiFCfGaRozd1TittcPIoGPsiBePL6XfEXh0MfPNkeun87VYhJWRC/Gj1zny9ScdRrvkQ1pZ4IuHt/LxmjleYMDgAsz6SaztWioFntbAfBoo0evL11h43hsY324zJhHGz3yiJTWqTgT1OLxvXRmdc3YKVw+vqnAVtnzaCsIcp99Ejfuqwh+cBdcxK14b2OhTue/vhNpidHrByPXz8bV4pLFqQrcXv2/HtD/uf7vkZYb03SPTGRRGFRkDH524/7I/JDnDQ4ArsyEGAoLD9sibipR4fL5sMtNnshdJiwRzB1OEw6IIoIozgQlro+NEJKPr1u8/LJHK795oxALUZpxL0XGZWzLbjCu3UO706WD579h09sFsrAiIrbzunw/OCuW+C7dMzXNaeJc09qksT0XGgdGetJN23UDYOrdZCoXiI0HixMVcXqxhON44sT66HbWSdwA8vXhKHue8vjCDTFfVcRHhZRVHos4inPiRKBNO5W8/aYkqMa2X/aUk4TsBtNdJ5MQCn++sVBnaw/J77MRqt1enz5YXRu5N9Na9oFo8Nu4prVpCHuuVx0A08BUW4aarTZ9+MV6JLtJZVGR8YiUheOI1CnCR/ftpmt3H1pnl8WxRHC79nqtou1ub8LkMqz4Za3AElariJtk64WbRGUJm3QXcVMat8nCNo19yrj0d5MbTGchtOHff+7Sa0tXqG7I4pTvh4sVS743SUph5J0k7X0AmHWmVgyJxYpL8xYWFdWEGhDRtbsPR44VXiA4AWKTcm2b8it/b63qk1/yRoJZ0/DZ6wRY3VKkmMRdEpdGVuhEoM3vntZFVRVjc8bgthLvV208bBBDWoyDj989QB+/e0DrknEVYN1eX3t+0+ASTdLeB4BZZ2rdZCaLR63i0/WlY6zLTEwsrmb1o/t2G1+3SfkNf++jjR6RNzjvNDNjaioXFw1agtiawk3NOm1dGuOs3sydk0dk9btnKc7IphlrY6FOW5ZCSNdIlcviDN+PcKbY/PPNgo5+EEx1VhT3G2oVv9ACHYBxMLWWIdOiJGIeTFYaVwuAbFHiXldliO3wR3UpZ03ZuX2O1s69rf1tLnDr19Nef1hYkqsWbGu1snVpjFNImO67Kch2lgr62bitiOzus1/2ImUswsQtWCpXQ+dIK+kgj3D3afmd/ZpPAQCICmoZsglcNS1KIubBZKXRdSpX4WIxeLq5Nfz/Rxu9EYvTuCwPPzMBrhu9LdYa5mq1sg06TyIk0iyaaGMNnKWCfrIVhmhg2REbAtM1kZmv+rTymzeMXarjjgNhTTIRTjo4dWgQ91b0QHjb+wTGj5ifXlu6Qn/3h/9Cry1doYU/fkMHz39T+HE3LRTOMmQbuGoTUyCq4O6q+LTDL1FnIxrMy+12vefnErYO2VoMTBancVkebK028rm5Wq1suhgnERJxgpl1vbKOXPxWeW8+/GKdzqyuDd9rimuZJsTv0l1nm/5jAu7ZTENQ6nqXEb1wARNNXyC86T4RmfvDzcqYHgfNVjtSI07ErclJCUUfd9NA4cSQbR0Nm6aaRDSsQF3xy/TJ6YORgcgVI+T68ti6FEyWH9vjJMUlENVUeVpnteJcGmlMuq6uzPB3h+879zvEJCYmrlOH+POdxkXF5jrbFDkMP5uisa9twL4J05j+5cnmcCMzjYHw3G9avnyHnm5uscJv2oThpDnbvB0pVKmj6OOu6BRKDLnW0RATs003bG4gNhbqbBE4lciy3R2bLD8uu+wkhL+npOg4Hz63NKxWaVYG1rkyj1z8duS62Uz4Ntaybq8/MtGFd9/TuKgkcd26ZGQmxbQRkss+TGMgPHfuqppPJosvFmg7wuP76L7dTkJIUORxV3QKJYZMdTS43bh4kFX1UmS4gciZ3TlXmc1Cb2P5sRUMSa0Q8ve8vnSFfZ8QF0f37aZLt9qZW61s0YmXsBCxmfBtrWXhiU7OiJvGRSWuCJ6ExUGM6deXrigXJPGsT2MgvEsNJiKzxbfd6SrnOTBANb7jCCGi4oy7abR8FyqAWqeaj+7brQ16bSzUqbpNr/24gcg1KgxoILDiVCEOpwbHTZWPW1GXCzg2PYztTpcu3WrTqUP13DTANAXuyiLFxhIQDkR14cdONzVrwzgrXNt8V9yg8TgV19PCVBZgGgPhud+krBRPoxZfjmmp0p0FXBNiV4oy7qa1inuhLEPcjqdW8ZXd5MO7cd1i5Jc82ni2qWyuqnOVPdroDa1Nrjtek+XHRn3HMW3rdupH9+2mP924rz3vbq9P1+4+zMzN4YpNfJirJUDcG66IJoc4TlJrwzitKbbfFdd1O0lXlMkCOy539DjhfhNRNHDdttfgNFg2syKNcVyr+LT8zv5CXN9pdacWSgzp6miYquQS6TPDyCOtqDFlqAjSGhS6BYroxURn0/4jTBo79bz5tk3iRbYEqMbQ0X27lXWVVO/3iOg//t1L9F/v/8wuLKpFZePZprW7Ia0JJ21BHSfWa5KuKBuxk2b8mopJuBR0v4k7F/Ffm0a5gml0l7ji6pZU8fjZZkpnkz3TGGdHVDAxpJvYwumLArm7NrcQ7vBLkViiuHEkROkMCtuMEA7dQpPGYM6rbzuOJSAcAyVKLtz84Se60DgQeb8Yc6aFIDwmH230aPGr9ZHz4EjjHtlafLKe3LLIjAwX/QwCYvvcuYidJIu76rNE+QqmN10LEVdn2y7I9NtmQSxxa0PJe9FqxoTc6y9L0rgfnPgreZ7Ss1IUCiGGbG4gV+Fffp0TUzZWpThZV0lwyQgJY1poTDt10y4nz77tOJYAVV2hgIg+u3GfDr/6EruA6BaWxkKdzn99J/J6rx/Q+a/vGCeKOP3rwr/V1uKj+640Js+0XVHhRTitei1JXJPcZ3f4pcK5FGzFq2l8zUqqfmOhTjd/+CkSNG0rhARZW1aS3g+5wruq0Xm49IjtcfNC7sWQ7Q3sMFli4ddVCxi3ExJWpfCC8P5be7RxNUf37aaD578ZCpf5qk/nTrr5g+OYXr3nnzMtNKbJLvw3v+zRzm1zE+sw74qr24ObhLhaUrZwmYu6jEaBzYJkejZsLT4612Fai1marihT30FbsRF+rjeebcYWLpww4M5z3C4FF1FrK17jNGjOuxB0Qb6mJc+LFTQtk7W13bZGnyBsfZUL5cptbcoKw0AR73PuxVAau1sdzVabHj9V+2sfP9uks83bEReKTghV/RKt/suDkerKLu4RAVfskaNeq1gHNIcnO+FmENWVTx2q07W7D3Nh2h6HmV0nPG0XLc5FEhebBSmtKubcd+V1MbO5J6b3qIRklt8XZpxu5marTYtfrg/npHanS4tf6ucjlXgNj/Fa1VcKe/HbpjW2hCg6fjgvgS1+2Uvd2h4WMy41+nTWV0FAg8BvzltRtPucezFk+0BxWVBcF3mi6A0P0+sH9Pl3D6wHesUv03a/RBu96OBw9QnrMtjCeKT/ndzxOVP2pVvtiabKC7I0s8sThRxXFsZm0eLOs+KXqNvbiry/pvk+GZM1xVRs0qUelOq7bNzHk8DGalryPG2wusm6FP6+uOdUq/iRGL9xu5mXL98Z2ZwRDQpPLl82u2sFqjHulzxl493HT+2aYBcZl/FjdbzfvJHqfGsjZgSq+2H7+3RhG0W7z7kXQ7YPlE23+DA2N9xF8X/87gF2ASHSLyIqy4JtBpuIb/nTjfsjLQ04a4X82uOn8V0DWZOVZSI8UXS6PSoRUVi22C5a3Hmq4tj8ksd2ETdZwcICzvOIuOEp14OKa+XL62Jmk8zQDwKtcLYVdLZjwNQxfpJBxNyCZRN/KFCN8d5WMGjO7I0utp1ub9iuJk/FWdPEZvz4JY9+tWOOHm30lDE2gnqtkvp4cBFrYde7rkSJLUW8z7kWQ81WmzYUKYdxen25/k2g8oeqEANaN5C4RYSzLJw6VDfW/BGE20Lc/OGniHtv8at1ooBGzOUck979684h6bmpJootGuzid26fG8kyW7l6b6Q5q8vCGh42utgxkxVMJeBMJK0HNa7+eK6o3LydjZ6yIni4ua74rM6SI48BW+FicmtOemORFF1CR71WUWbjXrv7cGqbGXPjp+x5tBUEkd/abLWVHRCyep5s58haxR85R9uMaRN58Cy4klsx1Gy1afGr9YgJlmjwoIlMHdPkptvFmsztFb9sJUjChctk/7xA5xPmLAvX7j6keY2vl6Pb6yvde6prySFnEmXRTNP2HFzuqW18ETdR/Nzt0dq5t4fHsnXR2Qa7V7fNObltZCtYXLN8EuGoW+AnnTIddutxbWS4DBedJSfu77CJs5mEGODmEK4itQpdjTZdvJ3sjhcbi5Wr9woviriwjPfefGVYjkMmfB2yHg82c5JsuSRKz/WXhaVrHOS2Hcf5r+9oF28RlCxKgMcpq69r4+AR0alDdbrQOKCdNMKtKBoLdVr57RsjcSHzVV/rE9ZZQM6d3B85R6aKwAhJAvrEBHdmdW34QIUXlXG0hhApnDLcPXUpEW9q0UDkVpjS1A5EoBMmpgDuuKImqUursVCn60vH6PuLJ+j60rERK1WeyvHb/E75/jUW0mmHoyMv1+nE3/868ppf9ujcSbW7VoWuJVGZqWsi7klerkOacOEXn3/3QPu7VM9TFqjmJL/sDdyapB7vaXgD8mA5jktuLUM21hA5KDlOLRPxtw+/WI+Ih4BeDPhzJ/crd5Hc5OmaRqyzgKh+lyowNoyte49oINaq2+Yi9SO4T2cZUxS2yMgpnDqrlEt8kY37x8VFF75HXA0qnUWLiymQ+0aZdnrhY8iB9WnuSPOYZWZbFFUEl8tZlFmRh+vUbLXp0q3RxdkjotP/4RWnc9AldPSDgCp+mX2e8nAd0oabH0yxauPCZT0Uc0PSR6HuEFqQR3IrhmwJF0aMU/+EC3pud7rDippZppvbVE0Of9fhV19iC2AJ915YMPllbyRmSLxXxLG49OHKKqZINXEKIaSLfTFlVqniN3QThauLTr5HKt+7bsfETUQevQhuNPWMq/hl+oc9u+if//LTiJgVC2E4fkw3YZuEUx5Tpm0FqezWcS3W6Coo83CduOdJl1jCwSV0iE0Kd21cns2ioNuc5EXo2ayHacUJ+SVPWck/D8LQltyKIV39Apk0Mlt0A1uYdbNMNzctzuHgO9lS8unpg+xnhWDSZZPZTFoqalVf2csrKXEXEN09VD2UpokiSfCwq5VSV/RRfEa3eMmLkSqIWBU/xk3YNrFSbPBxRmPCFpMg1WX0EJkL0Lm2ntjFzGHjzMbTCRHX1gm6Z0L3PLk+m0XAZInMQwKKDWnFCfW21GVo8iIMbcitGFp+Z78yEFkmrUJVNiZ23eKRhvuBm0xUgeThzLGP3z2gtJpwx4wzacn4ZY9+ebKpbWxrgrtucdO5TffQ9aGM43YNf972vdxvroeC2E3oTPe277dxaaiudRpjIk1U9y/JNTRdl7PN2yPtGNqdLvllj/ySF7HEjjOmwmajZ3ufxN/lnns7fHPYadrPZh7QhVgQTb78hCCulTcOLvNMHsmtGApPZrWqT096/WERuzgtLmy/y7YTfJyigK7iaeXqPW0gebgXUBJhppq0wiXX689rE4V3vC4TmmrhENctrkVGvodJq0nLx7T5PVlcdyKizsYz44aA6MX146rMcvFjqgnbxjKnEhpJx0Ra6O6FjRuYW8R016XZakf6UhENYhpFPN6krGVJNnocTzdfVOR6tNEzznlZPJt5QPyuPJafIIpupIflVchs5Y2DyzyTR3IrhojS7Wfk8l3cpBm+qa6BgXHEk20rgDSqNdtaQ7g0Zptz5RYOcd2EhSuOuBD3cOGP3yhFQc0hldiWNK97uMv942f25utur0/b50rKQFaX4ncuLTzk35dkTKSF6V6YhIF8TcKiagdTTXxXxdcGn3Y2etT66O3kPy4mcTd6HDZzHidIdXGJRVkwwyS1IGeJKiM73CjaFItoi1/y6PQ/vlLoIpu5FUOTrM9ha53Q+cFVxMmqsFHuL9cqqWVs2AhQ04Jp6qTOTcjt56IuqQjmMoTSzhxqttpKM7npunPXZ+XqPaeqwGF+7vbok9MHrePHVOcX1zKXh2rVpmdAZW0OAoo0IHbpW/b4WdQiJpOHRT7ORo/DZDlUXbszq2v0weraMNuoyAuminFu2l0wNYpWZRrGxhsk9djOM3kkl2Ioy55UNnBqn4hGAkS5gEyu7kacrIrF43vZ4pOCjWebTk34ksLFjDx+ukmvLV0ZuS62ndQFadznn5nFiXs9DmKMuvrJdWPb9l7pzNG6ODFb6xqR+053nNWqwwkFtYpPy+/st3bxmX6LS1Bprx+w90POBswDzVabHj1+qvybbW9Dk+hVdUaX54KkLWJAeqQVPE30osxNlrWTsiaXYijtuhRxrEzhSdNlt8gtkHGyKsT/q0q5C3S9b7LYmap22L88ebFD5lxgugBp1XvjMg4rhWki4b5LN7Ztq8Zm3fMpzk7XVkQltfiqEgo63R4tfrmeWvaW6wZCVWfHI6L339qTm4XhxfwVdfUR2afa60Rvs9U21oezbRGTh8rdRabZarO9C0VB4LQ3ykWM+5LJZQVqmx2eqFL8+tIVOnLxW7bqZ1rVT11UdJ2ZfE2VirkKx42FOrU+epv+evEE/fXiCeXxRXFCGVFTRXd94tJYeFFJtbptzhjkK+6dzQ406UMVpxq5K7pz1H2Xbmwf3bc7cg9VVWMvNA5kXj05DvKYUO0Q03gWuYSC3lZAnkep3HdX8SSuv3w/Pjl9UNmWYVKY5i/beaKxwFfuFi2STIgYR27+nsaK1eNEXD+VEJIbRae9Uc6DSzgJubQM2cSkhK00i1+t0/LlOxH/f1pWJtsF2i/x6f5pZVXo6tKIwmg6V1Xa2JyzuHc2O9A0WkgQZRvUqGvUqBMm3Od2VXy6dKsdsaptK5eU/bLyGqegI41nUTfWOht8zJQLttWsiezq7OQBm2c0aTasbQ/FgIjOrK6x89M0VqweJ5zwLXserfz2RVsoro9mHIoe90WUUzFkMsWqglZ7/WBoIpdTCHVxOiJY1wbrFERD47A0sip0dWmuLx1THltXJ0kVf+Ey6ZiujfygmCbltB6qrBcnboxyQkiuF6SqGO55pJzAHj/rF7IonWrRTKMis26spbUzVYnpo/t207W7D2lSTYuTYjt/pZ0Ny6FzpdsFaf/r0OVX8oj+lzf35MoSN0m467cVBJEQDF34hS1Feg505FIM6QKYdUGrMiKFUDcJ2DzIukVMFacj90vTkSTg1LvxmfIAACAASURBVPRZ20VHF39BZD/B6eoThR8U3f0oykMlxkS317daGG36rXEtYYgGC8UHBer2zS2aXB0kFxHDJRSIdgBpLdZ5t/S44mLtilt40rZrgOl7dZ6BZqtN/3l1jeTIp62AhunhEERuMZOdhELo929NjwjNZcwQkTr+wDX6/dFGTxunw8XoCGTfNdGo8KnXKolqduh870k/yy0uqjpJXPyF7rrYnM8npw/SX0OdznWd6D89fbAQmQjhMSGCZ3UixdRvTQSWmyhK7AS3aAZB8piexkKdVn7zBs1LNaNqFZ9WfvsGXbv7kF2sbbCNQ4xL1sfXoXpGRSBtmDiFJ4kGXQP8ksE0rmHX8/NRxc6JcbJy9R6pQ8AHHeOBW8xkUmtqnB53eSWXliGOOIG1YoHiOi7LptewJYoTXyJllov9sR1gSXafqmw3kfa/q+KTX/ZGhI7qYdBdzzSrNcftRJ9H4sQz2FjqbHfuNrETWWfixC3xr6uD5AI31jjrmm0x0CzLeUy6XIj4Hl2GLJFenJosDuGYSFMvuDCPn23S2ebtSOycR0SnDg3OXWdBtfEYzAIuMZNJiy4WPYNMJneWId3uyVXFip1PY6HOZngJ06sqe0HX00e4LLLOWrIhfP6dbo96/YB2bitrrU6665l1GnrYMlIU4sS92FjqGgt1OnWobgo5M35X1pk4NsffxVgcdlX8ocX3k+cNhs+srqVmJbG1iKrQidw0iHP8rC1JrtZpm/lO3N+/XjwxvMe29PqDZp+quUJYIHT3kqvvNouYMjsFSS07Rc8gk8mVGDJNtNzD+Pu39ijNs51ubziJ6B5kbqLSPVw/drqJXF1x4CZHzoK18axPn2jcT4vH95Jfjv5GXUZcHHRB7K9pJvpJuhU44iy4NouIqAZrs7fVfZfNopvkutocn3tsxOtZCbYkm5M0grvTPP640sttF03xXpf5TrcJ5TAVMV08vpddtN578xWn7wLJxvc0ZJDJ5MpN5lpOX9VugEsr//jdQW0W1Wc5F1o/CIzFDMcVaBmncrGwYOkmKyJKnE1mwpTJoiqNEC7bP+ku6II4ge82ZmudSzaceab7rjjtElyuq82izgVlitezSp1OUlIh60KdrsHjeU0vd53vuOdl+1xJGWxtavYpvhvZZOkQp1Gr9/xzRQpvsCE3YqjZalvV3uEeRl3KutwEVJVazgke0Usn3Fh0nIpYzmYLY1O52KT8sxJzclyJKoYpTLg0gq6Z66QXAyL3Bdd0nU21o2y/y5SJE6eXmu3xbd+TpRUm7njmYidEkdAkcVjNVpt+ebIZed0v8xbYrC1V48KUGWzTVNij0WKt05bpN0lcsgyJBpvltXOTazycJblwk4ndKofL7sx1Elm+fEcphESQ9IXGAfrk9MGJVPsNZy6p+LHTpcXje9lYk0n4dFUxTE6RlMS/PQ+LgYtrwRbuPom4Ktvv4lxFIu3ctZea7fHlRd30niSxPVnBxU5cu/swsctq5eo9ZWG7ndvm2PuZx2sUl8ZCnRaP76WXn4t64VJVudwuNA5EYucCIrp0q50LN/m0IVyftkxzWFYuxJAuZd7VCuMyiTRbbbYuRkAvdjVZLH4qwrEcy5ejTQ/DiKDU99/aw6ajjhvV/extBakEOBZxMbAhrWB8Lq5DlXYu45IBGT7+qUODshdi3BKNLnTzVZ+2z5WGwdJH9+2O/NYsW8fYoNtEJQ2u1mXXceQlOSMNODFJRMp59drdh0qr8PJlu3YfwA2X2K6kdYnyTC7cZLpd6alDgwdE7havM1Fzbq2j+3ZHjqGbzFwD/3TYmNhdGsHKPH62Sc1Wmy40Dgzjpibd3JC7n6qGljpc42WKTJJ4F9Wxwp/TpSTHqfUjjs/FIH387gG6vnRM+Xe5c/m4W8dw6Fx7SV1WceKR0hwPkySOa5a7rp1uz6lrALDH1l02rZtRIiIvcKjNcPjw4eDmzZupnwTXmoJokNlEHo3Em4iO0OGAOVXdDI+I/uPfvUT/9f7PEf+07sZ/evpgJvVFxHeHXW26a2BCuFLyAvdbRE2h5ct3jJVqRfxAuAXCfNWnIKBID7oiMMlO3Nw9KXse/dPv3rA6D64Wl2trGJe/jwvdc2r6jUmOXZSxGwfV75bxiOj7iycir+vmwrzNddNEOM7z8bPNSK26Io5Zz/NuBUFw2PS+XLjJdFWie1tBJPA2IKLPbtyPmNO5ejY3/u2RU+r8fNVP7YbbmtiTxMLkIY5GRmfi113XsudF4gfEscTO8tFGjzrd3tDc/sHqGh08/03u4wkm3YmbuycuQsilFpcYkyarSl4ChXVp40ldVuMuwZEXTB0DalV1Pao42ZLAjKmkhhwOsnbubVr5zRszNWZz4SYzVYlWoUob17lnuNdVWU4n/v7X1udhwnay50zp81Wfqtvm6MdOl0qGtNO8oDPx6+K0toIgslO0acHS6fZykXavY9Kp0kndLtz5l7xBb6gwYkyaXERZp7S7oMtUJeLHM/e6bAGdr/p07mS6JSvyjkm4cE6JxgLfQJQTUEBPnJIas5a1lwsxRPS85xBjjuawFRRc7Yr5qq8MYlz9lwd0+NWXUssSspnsuXoc8gRqWz5/ku4Y0/fr4rRUC6DtTjAPafc68mABSTK58Z2wo6/JY9JUl0n390mPYxnVteMWmJs//ESr//JgJIPs0UaPFr9ya4BcdEw1bHQB5OdO7lc25P3lySbihmIw6c1YEciFm0ygMkfrGv+pBIXKnP3em68oXw8C9WRu06g0bHI827ytNEHamthtTOm694jzeW3pCp1ZXZuYO8bkDtIt/kkbCebZhF70VGnb85T7SBGZxzX3dyKaqFuRyOxW4BaYz797oEyl7/XdGiAXHV34A5E5gHzntuhe3bWJNBiQh81Y3smNZYiIN0ff/OEnY+FDsYsUsUD9IBhpAipXqC57HnV7fa37JTxI5F1qrerTL082hxNeu9MdKdimMkHa7HBtdu42O9RJFis07UB07kCiaNagS1GwPAuLOJWr88Ti8b1WbmzRR8rFqqMa00cufjvRnayNW8HVLa/7zDQirpMqYcJm7HOWo1m6hmlh46HIkyV2EuTKMsRhKnwYLk7YDwLySx5tPNsc1jYherFTseluLCr2qqwtjzZ6yp2fTJpNHk3YxNWMawLR9SF7fekKPX66GemHVvHLdOLvf83WIgnXrKn60WGbd2FR9CDaxkJ92PjYhBxcrbLq2PRGm/RO1ibxgRPfunpaeRbsWdBYqNPaubfpU8fCtc1Wm0rMdZy1a5gGJg/FpBM88kAuUusFZ5u3lRYg2RUkW2ee9vq08bw/jYmKX6YdfkkZlBfGL3l0+h9fiZSFd8Ujok9OH0wtrZZT7q8vXTEWeB5XSurB898Y0+b9kke/2jFHnY0X6fFcvFjZ82grCCI7laLtYsJpq543ENUqK2ZeMaVKC7gYPfEbk5SaGNc41j1Toj0Kl3586lA9EjNENGi/sfIbu+y9WUY3zoqa3p0HdHPmpJ+3LLFNrc+Nm6zZamv7URGN9rKxETXh49gIG9Go1Kb6s4mXa5XUAtfCQlG2nJgCFdPuQq/Dpsh0byug6rY5an30oseNrlkuUdRNUaRMh/DkLotF7vflkbDLlxMD3HNjquYs/+5JuxW5Z0pUyiYa3Ee/5NF81R8R9uJ3yM/rzm1l+t/+ZyziJrgijUQDkQ0hFB9uzrTtCzrt5EYMrVy9p+1HZeMKiktY/erSv20REzdX+VceZCYrh0kock0mh4yxn4xtuXb5959t8n3pZIqa/WA7dru9Pn34Rb4zjsITqkshRpdqzpOqwCw3Rg5XQFc1dFYJ+2arTZdutUfea/CqA3qxaeDCGLaCILfPRVFJsy9o0cmNGNIpUN0k6kKt4tPTzS3jbjNurI/KpaNbGIjsAjVNQpFrMikQWSzjmEhMVir5fUQvhJ4tRdypuJxzPwhybyGS4XabnFXH9DzYHDsrdIkItYrPbpDC9xdpzPEwbRpmaWEeF2n2BS06uQmg5ga66B7v8iCoDCEVv0zL7+ynU4fqwwDHsueNpAGLwE7TYl6r+Mog4H/63RuRpoOmwDWbQM00hOK4RIQpnZYo+vtdNs1FnBBdz3mcwfdZkGU15yzRucafbm4NMx7DhO/vpIO/i4ru+uRljEwbums+ay7J3FiGVDECogeZuCGm4M0SEZUVFaVF9Vcioku32kMzbD8I6NKtNh1+9SWr4xO9CAIlsk+X173XZuLUxS/odtsy4xIRNtXE5YfMZYEo6oToUh5AULSFM+wuO7pvt/J9abm/kgbQq85X5xrv9vq0fa4UiYlSjUlTGnPRgv/Hha5o7qwtzOOCu+b1WmXmrnduxFB4kqw9b8j52Y37dO3uQ1o8vpdOHaorY2eIBtYakaET5r93N4fH1llhbBarcFdugbAqyZOrqiv3xrPNkePZ1H9YPL5XWY3VViiOW0Tc/OEn9m/hh8zWrSYC24v4gKoCj7mxKpiUBSzOQq1y9erqbtm6v3StLlxbC5jO18ZVK8SSKQOQ29gd3bfb+txnsZ2HbaYhSI9JJyrkiVyl1gu4thNcarwIgNalw+qyXITDy8VdIwdd26YcC+QUW+6z8uLfbLVp8cv1kVRdv+TRym9fpOmGyw5MqrN7s9WmM6tr7LUUwrDsefTem6/Q4VdfUv7+ql+ibm9r6nfOeepoHvdcbFzLRG5pull2kbc9Xx2m66IqE6IKwBaE55Pw8040G6n5sJqNn2m/5oVLrZfhLDic2Gh3unTk4rdaMSNXpg4jduFx+6K5ZrrJAc1i0IUbE8rNR1eu3otMjKIsvdzWIA8D2BQDJP7WD4Kh9UAscNP6MOqYVNaUiriBv7YuPRfXn+5cksbkpOGCNF2Xa3cfRp4D3XMRnk907Tym+dnIyzw2S+CaD8ilGHKdrOTaHzr6QcD6/G/+8JMyPb3ql5SFHWU3RpzJVf6MyDoLW73SmvzHies5ff7dA7rQGPSiEr9VuC1n5QHNy2QUd5y5ZhAmPZekne51MXi2lhzdOZr+xp1TFscFANiRm2wyGW5Sq1X8SCaKacKSEVktqiwXLj19u182ljHnysbrcMlAKVKTT9dz6gfBzJSCt2lDMUnijjPXDEIZ7proziVpRhr3+fff2jMyN3xy+iD99eIJqjPnEhCx99HlOQifu+6zeXzmAZgGcimGuMlq+Z39ETFjK4TEhNNYqNP1pWORFHhOjHQ2etpO8boiYRxlRUXoLCf/cWKzMMqUPc+qvEAecRE3cQTfuMVT3HHWWKhHSlYc+buXjL2omq02LX61PnJNFr9aH7xuOJcdUn+6WsV3irHiUv8vNA4o5wbdmObuo+1zoLo2i8f3kl+KbrD88vgqyQMwa+QygJrIPqiLC4acr/pU3TZnHYcRpzdL0kBMORvFFLxapCA3uYqviNPi3I2/f2sPmyHoEdH3F09kfr5xcA02dh1fpj59WZFGNpntuS788RtlQsR81afWR28rz4VIXdAxreuiy2DTlbCoVXzauX0ucq5cNWvTOc9iNhkAWWAbQJ1bMaQj3PQy3B9JTDwuzS/jTOg2DVJNFFXwxOFs8zZ9/t0D6gfBMJvsQuNAIZsEup4zN1ZUgk+XkZfHaxL3/r22dIX9218ZEZzlWLGZA2yf+fDnpv3ZTgKuDciSQmeTyaiKo8nd5OVmiY82eiM7MJf6I3GyemwDR3XIWSl5CaTNiguNA8NgaZmi1bqI09jQJejX1H7FFU6EpsU4A/yz/C6bbDrbZz78uWl/tuOStGYUAGmRy5ghgSrO4rMb9yMTlmiWqIohcok94eKJOFRxAXEu6CxkiOjiX3TtG/JG3MaGLvE4pvYrLpxt3qY/3bg/UnX9TzfuWzfHtSFu4HWtom5vwb2e5LtssBFaLjFx7U43t8HyeaGo8YJg+si1ZUjVKyjOjjkrsaGyJm0829RWFlYx7RkiNru/ouyc4zY2dLE8mtqvuPD5dw/Y19OyDsW17C2/s19ZTHT5nf2pf5dA55Kxsd7J99HGQiQHy8ufBwN0AhTuMzBOciuGmq22tldQGF3hxCzFRngRf10TB6Gi4pfp6L7dI608pu2hn6Yu3jphLTf9VWEr+Gz69NnCZTq6ZkDqiFs4Ms7nkhSpNIlyrodcu9Olv/vDfxm6F8V9dEmgKOp4zxpOgNaqPtxnYKzkVgzpzKSqzAxdpkmc2BOXXYn83hJT5VpFXREDVfSHXnXdilQ00oQuZuTz7x7Q4VdfSnzf0qxKzVVdL8eojaXDRuhxGVKugc9xrYgmUa6z+sgV04VFzbUBbxHHe9Zwlr4giPaKhKAEWZLbmCHdxBEujibiS9KKPXGpCRN+r60Q8ojo+tIxunb3odFnnvdifQLuuu1iYkCK6B7UCet+EETGSdx75xq/xvHem684vR4X0+8U/bZka++jjd6wrtA44OYUObaHaPBccmJRuB2F6BdtfogG882npw+yRRrD470oz3WWcHP2z4xXAIISZEVuLUPcDny+6mtjHWx3jTrLjymoLxwj5NKXTCAmRpPVxDXbYpJ+du667fBLbBuUotFYqEf6yMnIu9c8ZMqIZyXLbDKb35lmv624Y1xn1QvH9ujci+HfK9r8yOdhslDnYWzkBdWczcVkFXEDBYpBbi1DXPbNuZN8cKUtJsuPbgcZ/pwuWJrbXcqBsKbsGJdsC11F33EQp4p3ETl3cr82o0hch7xkylxoHKC/fPyf6K8XT9BfPv5PqQohIv3vFNYPXWyNbrcftp6cbd6O3brFJhNMnDf37NpUTLexUOdlbOSVIlXdB9NBbi1DWXbzNsUO6HaQLlYgVWPYcCCsKTvGJd7m/Nd3RopPEg123ue/vhP7urnswnXZOEXJFrNB/I4Pv1hXWhBKnqctzjdtpn7T5sH0zNSqfiSBgIgiFjhRWoMrn+EasK27P++/tUfZuPm9N1+hzxSvi/M7cvHb4TMin48QdbpGs+K7dUxzhlX4t506NOgZOY2/FeSP3IohonRcXipMAsM1MJJDVMDWnZtJ9LkU6+OsVKrXba6Zqym/aMUTk8C5Q4jMcWPTZurnxmjZ86yeoZ83esMx2u50afHLdSKPIsKeKF5pDRl5TuEsVi/XKlr34rW7D1kxo3pGVM8R12BaNzam2bUWbj/T7nTp0q12oS3IoFjkWgzZEGeCMAmMsEAxZYhV/BI96W0pM9xsBJ3uPVkIDNtr5poSn6U1L4+4jhOi6RSH3Bi13UyEO9apYotMxBGYpmfLpWK6TPgZUT1HAemzYlVMskRFlhapZqudyOIHQBoUXgzFmSC4Oi6cmVtXO0j0IBLnoposkkwkLgKjVvGVtZnCFX1tr1mclPhpcofZYDtOPKKpFYfcGLUtTOhKWET4JY82nm3S60tXnK5xGvWRbNxd3PMi+iemZdHOiqwtUmm3nwEgDoUXQ3EXbCJ1R2nVg65zA8hmXC67SzWR3Pzhp6E/vFb1KQiIfu72WFeazaRjW9HX9prVnvd7CzNtbh5bTKKWLSBX8Wnt3NvjPNVM0P1+boym4W6WqfjlkVgS0ahZdrO5LNRxxbup8KL8jOjGhQsuLvM0cdlwxqnPphPMszrXgPGT22wyW+L2KhJ1XLh+Zh9+sT7MYDm6b7cys+GffveGcSLlJpLPbtwfZsQ82uhRp9tzzo5R/aaV374xksWy8tvoOdpcs2arTb882Yy8xy97U+fmsYHLQDzbvD3Mdtp4tql8oDrdXqq9wCaBS+0tgSqrar5qFgB+ySO/HM3mqlV8+vjdgetK1GDauX0uEls0zqwsm6wn1Xv8kkePn206Xc9JZVjZbp7i1mfjiNN+BoC4FN4ylDSmhnvQReyHCOSLm9mgM5FzJPGV2+x0ba4ZVxdm57a5qXPzyHA7W52oFVdJV2bhsxv32erURcgQihuvosqqCo89v+zRzm1zI5ZR8Z2mazLp6uY2rjbVe1Q9DMM1qnR10MKZd1m28+EsxLVqPPc7916ZuO1nAIhL4cVQXL+/mGxsQjW7vT5du/vQuW0Akb7Qm44sJ3Oba8Z9P1cZdhrQxUbEEbXh93FuhSJkCLlYB5JkT8okKaQ4TvdKnCQJLr5MNCjVjQmduMxi/HA5AeHXXYSpbn6r53RDAKabwoshIne/v2p3aiKuOOGCtU2LaNaTuema5WGRGTe6nW1cUSsjjyFdvEQes2hsxoPtwpxmkH0Ryzk0W20287DkeXT+6zuJLCxctfy4AoPbAIVfd5kzuPfWa5VYm04AklL4mKE46Ey0JaZ/ZVwRoIqbeP+tPdpKuHmYzItWATaNPk+6na3qeri2OhXFBV9bukJnVtdiV2WeBDbjYRJVlVXPV5q1adLuHyYEo67dB+duFdmu8jm4VMuPG4toG5fpMmfYvhf928C4KKxlSN5Zi87ctuZV3UJT9jzaCk1USYOGVTvhw6++NNy11ao+Pen1qdsbVFzZ4es16jhiTIpUMygtVwG3WxUurnDc2NF9u+nSrXbEKvEPe3bRP//lp9H077JHvzx5EScyacugK0lcq1kLu6zKOagKASZ1QZliZUzI50BErIVJVfAyrsVRZX2TyxnsqvjkeYO2O7WqT9vnSsP4r6P7dtPK1Xt0ZnVtZMyEx5M4xpnVNVq5em843xbBhQymAy+w7LJORHT48OHg5s2bGZ6OHTo3l6j7o3tYuHTYMjOxyKnRaQiR8DG4RVX1O1S/XVXrSJ6gbM6zCEG8KpqtNtsWg8gt/sDkPlXdE/m6ySUSwtf/8dNNZQ0o2+8pAgfPf8P+xrzHgaieSVUhQKJkrhxdmxYX5qs+PeltsXMgN4Y9Ivr+4gnr71FtOmvPyxmoKoSL7xfzETdXmYLqPRpsCsUGUQauNOCC53m3giA4bHpfId1kut2VjVmeM9FyC6rwjZtSR21MuqpjfHbjvrV7gXNFLF++M3LcTnfQ4sDGRB4nbToPmFwORG6/RXa5qFDdE1Gi4ZPTB+lJb2tYIqHT7dGT3hZ9cvogXV86Zh14XuQmtkxvUyLK95jinsksCgG6WPx0ZQgebfSUc6CofcaNYZfvD6e/i16LHtMqRSCeE1u3KVehWyWEiPLnQgbTQSHFkOlhMP2dizMwTSCm7tzhCXXxq3U6eP6bEXHEPfi2v4PtDN9VT47h81RhmrTy6re3dTm4xK0IccOt69z1N11Dm0Xo0+fCqYhCiGhgBdOR167sLs8kUTIXpmojxlHdNsfOSRxbQUCNhXoqMX/cmNaVkBD82Olau01dxU3eXMhgOihkzJAps8fmYbGtmGvbQV41cfT6wdBtYNvB2/Q7kmQ1ucZ02KT5ThKXSVT8FltXoCkzJnwsU1sGUz+rWsWf+PVMis3YzOOu3vWcksYPEpHV2Gl3usoq1RW/TNvnSkqXJNdfMY7rOw0LmE12mcucluckDlBsCmkZ0u2ukjwspswUXVaFzcTR7fWpzPgSwq+6ZGGoqvVy5+nyeq3q04dfrI89Q8gWlx1ireo7uQJ1O2uVFZC7A/Li9PG7B9jFLdwypYjYWD3yuKt3Oaed28qpi1bOHeYRRQTPfHVQhft/euPXys8c3bd7+P/Cyvn9xROxLI67NO1CdDOOeE5srVOLx/eyx5uv+pllCgIgU0jLULi3mGs2menY4vNi9//B6trwO1Sdpo/u202ff/fA2LGc6IXfPWx9sq1wrdrx2QTn6kSiMlvkefYT95vS2OEnDdrmzpuC0c7nFb9MQUBO2TWNhTrd/OGn4X0tex6dOvSiH1WcLuRibBU1WN2G7XMlbfBuHnf1JqudzONn/UTVnlWWVtF+RI7D4WqRBYG+Qey1uw+tz4U7PzE2dTFgcpNZU7KGaayLZy0cp+XRIDaqum2OPjl9cGqeEZBPCimGiLLvjh6etIQokBe9upQJZiOEiF5k1SRZDG2r2RJFu6Xblvk3CaykO/w03G+69gTh186srimPwYm6Zqs9cl/7QUCXbrXp8KsvJe5CnvXYnQSmTLw8t1ewbZdBNPgdQojEGbNKd/rWIENr5/Y5o+us0+1pn8v2c3dwnOscvoe6Kc02o8s01uX5SIiqRxs9Y/NsANKmsGIoa3TBuWLRu750TGklEJSISM6HEJaCtBdD22quLmX+dQIrjWataXXC5q5lOHWXq8fCibo41ahV1zvLnlHjxGTN0j0veU+rJ4pahJcv34m8R2WtCY9Z3XVqttqsyPm52xuW7yDiy3/YEFc4uNRASsPCF56POt0eVfwy1Sp+RPDlsSo7mC4KGTOUBqYMKduMNd37dlX9kRgRUzFFl/OT4WI1Np5tjnzOpUKwzvKTRrPWNDth666VLv1e5zp0rUYdPlZRyxWosPkt3PXyiAqVISd+qypWx5T1qbtO4m8cNtWcbdHF9OmeFVvXd1qLBjcfcZavPAbfg+lhJi1DJguJzpIgKHkeNVttrUn70UZvZEJ7tNGz2rW5upDEa8uX74xMJOL7bv7wE127+9CY8SSzeHwvfcC4ltJo1mrbx4ibMD/8Yn34b9214na7oh5LnGwym0yduF3eOSYZZ6S7B6Ky8C7Fbp5ofAHTaV0fbrxUt81RdducckzsqvisJUcWJrpinmFR7uK6U6HaVJz/+s7I58PPim1W1xapmw67gpR6kCemWgyFJ4Baxafld/YbLSSmQn5EgxiSP/z5Np06VGcLtMUtiW+zkKom/53b55TmZV0BOSL1JNNYqEcmT937XbFtsMlNmP0goMUv12nn9jm2CKUu0FTUY0l6fhxptqYYd3kD27IB4hlpd7rklz3yS14kcH0cAdNpXh/dffvk9EFlW4rHz/TxdaZ7zonysAuYq9SserZ1DXRl5HnFJZA8DSsNN7ZU1bWRUg+yZmrFULPVpsWv1kcyNDrdHi1+uT4yYctw9YI4ur0+Xbv7kN5/aw/96cb9kb+Fs0Nk4tZiCZvjw5O/LsaJQzfJnDu5P7OO4LZ16qoZqwAAIABJREFUUHSLcW8rYBchU6BpyfPo9aUr2gwX7vxsFl+XDt4m0rYy6VD9Nm7Blen1A5qv+lTdNjc265XcKiJMnOtjii2La63R1dypS8c1ofp+rpWPqYGujJhXXBIp0tgQqVqeVPwynfj7X9OVf/334TmLTSwRZRKDd7Z5eyRr9K3/cZ7++v92pyLWD9gztWJo5eo9pRjpbQVsDzLbekEyP3a6dPjVl2j1Xx6Miqxg0J5AZWDiag3J56FbSLnFkftdHKbA1jQKt+mwCSR32a26IFs1OCsCd3424iSpZUlmnA1QXasxy3Q2etT66G3zG1PAlL1G5HZ9bGPLXBINwp+1HQ+uCQNy02eXBroCWdjYWKNcxjH3W0S2Zvhq/w/zOyLi7unmFt384aeR19Oyjp5t3h7ZyPaDgK7/5afhv5HJNjtMhRhSPXC6CYCr9SNS3lVCRGeOXrl6L2Jt4qxP4vt1mBZSnetI16RRJq3U2KwR383FLxFRxD2jQyUYXa0INuIkTSGZppXJhI2A4ES3rkhf2thYcF2uT9qxZUTqzYZpPMRx+ZmeUd05moRNknGs+y3c9f5v//fjyGvdXl9Zxy0N6+jn3z0wvgeZbLNB4cUQ98BxAZ1E5lo/qriALSLqhxZcv+Rpa9hwmPoNmSYgXWq3/LtqVZ9+3uhRuN1hGqnx40QXv0RE9KsdcyPuGc51UddY/lzqs9iKE90i5RLwqxPHaQdWm4JoPSJ6781XopZQInr8PHtxHIuGydXsaoXjxkXc2DKVgLLZWGThEuWsq8L9ZDquS60geQzqfourVTOr4q+2lnRksk0/hRdD3AO3wy+xcTtH9+021qexiQv41Y654UOfdiCgbgLSLY7y545c/FZ53mmkxrvgsmBz7z13cj9rHQq7Z3SmfV1Qta05PKkLLG62oKq4ZNqB1Sa3ZECDKse/2jEXGVu9fjC2HbTOJRynrhEnAkXW6LhcyVm4RMPnWKv6FASDrFCRNBL3fHVjWfdbXHss6kIbkmAbWoBMtumn8GKI7eK+0aNPTh+k//X/uE2Pn41O7KKSsK3ZmYsLEF26ucXx3MlB0F9aE2W4WusOv8SWwCfir41LanxSIUNkv2CbREK4dIBAZZEh4q+7TWaNjqQLYJzdv0ocqwp+JrUiiM99+MV6rN34uHbQugXMxv0bhhOBImuUKL6bSgX3XGXtEg1oMG+lVd05TnHSkuc5CSG/5NHpf3zFGChui3ztd/gl6vb0YgiZbLNB4cWQqR7MytV79PjZ6N9dFwzTBGVaHNPYKXPVWnU9e5JOrC4WDO69g8mGrxMUjqfQLe7L79hnt5ksf5yVSc7Y04mdJLFUae3+swqsFr/rzOqaNm17XHFMRNH7MV/1WVdoHHQiMO2YEd1zlWbgPfd9piraLriWIiAyu6Yi8ZneIEjcFChuQ/hadHtbgyKSHtFWQMgmm2EKL4biBhu7LBg2E1TWgcZxrAlxJlZ50VGlGXPfyZ0f525R7bhN9yotl4TOtflyrZJ5XZ+0dv9ZWhEaC+rmmXGyo5Ji29w06fc3FurOPezioHuWhVXLdYzrsrZ0Vj5B3N+nK7YZfl6Jya4NE35Lrx/Q+a/vUOujtxM/f6prv0VE9V12CSVgeim8GIobbCwWDBs3UNpxAXGII+pcz5trThtGdT3jTKZhYWWzuHOi0zWQWCcUs67rk9buX3UcjwYxcWlwoXGAiGikBss/7Nnl5KpNA9X9UDU3TeP7x5G5ZyP6XX4HJ95FOnpWMTHNVpseP9uMvC4SS4hGf8trhjIEOh5t9FIJzh9nmQpQLAovhojiBxu7WACytvwQ6Rf0uJO0y3nbFpz0np+rfFxuh1ir+PR0c4s9brvTHRZS21XxlUHvj5+qs5TkonvhLtcfrK7R8uU7bLaMTihmbR1wEammmjNh601AozFxSbLNRC0YsZiGa7DYuGrTQBf7Jjc3jUP4+tgUMUxK2oKLE++qdHQVFb9MR/ftdi5oyNVy6wcBnVldo/Nf3xkGaqchJtPYjIyzTAUoFlMhhsKE23BU/BLNV/3IDjaLANQk56wTZlnEEoSxXewDGp2YdDtEUTmWM9V79MLSJMTU9rkSPd18URCg0432dDPFQXCfk+GE4jgmTBuRaiPWr919yMaAECXLNrMRx+N4XrK6H6rre+lWm04dqtO1uw8zswKn/Szr6o6Z8IjoH/bsilXQkC9HMPhvuA9aUtLYjIxjHgXFZOrEkKoNR7e3RZv9ILKDzZPJ1OSaGYerziXdVb5G3A5RlB4Q2PZVkoWQILzo2lqxur0XfcrScKGNE25MfLC6Rh+srrFBxER8axkueJ07hg1ZPy9Z3Q/u+ly7+zDT+JG0n2XuubVJGw+I6J//7adILI+NyHVNj09KWrFwRJMNeQD5ZOrEkK4NR/jhzpPJ1LaqcdKHVuc24WJQTI0gdeUN5HMnGp2EXCfSHztdbT8qDrlPmW1F3/C5TmLCNIkMXU8sXWsZVfC6yl3E9elSfVeWZHU/JrkZStPtzonFU4fqEZefCu4Wm65DGq1ybPreEY2K36SFRscR8gCKx9SJIZcaKHmxABCNR5g1W+2RRrXtTpcWv3xhJVAtOjYxFHErMh+5+K2TqKlV/VT6lNnsevMwYcbdeYsK4zrRKFuIiKLutHDjYY5xPS9Z3I88bYaSoBOLcjq6rbgV2MQjyt/renwiOyFU9rzhM5tVjzKi5CILFJupE0O6BSRcTTYvFgCi8Qiz5ct3lD3Uli/fGbkm4d9vqu/BdZ82BWWKViaqCTG8Y6z4ZQoCSiyEBEXIHjm6b7e1KJGRK4zrxKOwEKlqQakQNVhu/NujYXbZqUOTF41xydNmKCmcWJRft2luKxO+DpxYEMc/27wdmQeS4Jc9ooBGNm+q46cRt5Z1OQ2Qf6ZODC0e3xuJGRKo3AN5sACI8yDKVphxvdpUr9vuklTdp22DMlXZUEQvTPzhIFZdDzjRgoHIrpXKrorvnD0zbq7dfRjrc6LCuPg9ujozulpQYfpBQP/1/s8j2WWmau55Jk+bobi4WDNUv5d7PmoVf+Q4JrHAdaGPg0eDTe3jp5uRuYk7vuxCT7MqvGw9LfI4AWa8wMGsefjw4eDmzZsZnk46hLPJVMTpX1R0dHU+/nrxxPD/ud5equaTnKuLC96s16LFzWwnMe67VMfU/Ra/5BF5FCnYp+tOPgleX7oSa3GRr4fNs2CLyz0F2cON7V/tmLOu/WT7rJuePVeXN4c8llzGP9cH0vaZ1n1X2ELlemwwWTzPuxUEwWHT+6bOMkT0wtqjG+CzaAblso/mq/7Iv3WZbeLvpiBol75WttY5zq2hc8cJ65NcONAve7TRG81Yy6LlgtxLbb7q07mT5g7hMnFihsKBpjZuEVMtKHFc7u9FcDlOI1wxSvGMc3NcePNhU0rAFGxuGgMqQRHGNhbR1oVu80yLa6ETXSovw6RKsIDsmEoxJDAtJuMe0JMK0BPfqxJCftkbNpQVcBObmFxlUzmXDeLaZTpuJfBwgLdcfffa3YeRc+wHAW0wjRnTWtTDgepEg8yvxa/cTO622Tr155lj4nqsXL1HZ1bXrAJaK355WAsqfF3DC6SuhQkYPzbjNTzHcbWVTFYOU7D5/9/e2fVGca17/qluF9BmSzQcIU3SSthMLoKESLDgnESbK3KRSMMBWSH7oCj7Q8BEloiENmYLTSyhPeRDJBORN1lw2FJyQa4yInPg2N7Io+RiTxKYzr5AB8wI3EDbrrkwq71q1XrWS710V3f9fzcJdruqelXVWv/1vJrmWtWF3V7q9OYH8V+dpd6UJefqQlfHSJ5nmuMhPXy8YhRoJrAJGC1GWgy5LCb9eqD7EaCns0YceeW5RDaYEAecq9BUt0Qdy4j0OzVdWi8XnJqlEjhXOFOtyuxCU7GQpeXC1z9qJ9ju6nqwumyFsX1XcTzTQiO7xFzaqRBtxGX4NhQelYBjV8qcYeRqOVRrgqWxoNiCzbnfqyJLPYc8vsLynCbBxUWoq+9HVtcxNgGjxUiLIZfFpF8PdNH9rjhrhC4bSQghLs6Dm9g4USmOZ0rrtU1kurFxKZbIidk0e72Hj/VtP3wxCWxdsLrpORDij4vtkIWIayHKtDE+oxBw7EPZM4xcLYcuNcHktji6+2q792meDZfxzepCT/N+6KjRekNX7thg+BlpMUS08TLp0j6LeKB1O0kivhx9XpYpzhrB4drgVZi0O91V7wBaNa1XuG/UiZIt2uhQLDFtLZ5A00G7u7beHTvrQpe2oCSRvvihcAlwjVF9ClHm0dm9DEKgHxS9gcmKKkC2NUJ69HQlkRjgGocjfs69a7Z77/tspBlfW3q/SYxlmWu3jYc0vinfpsCgXIy8GCLi07/zrpGi2+lMfb6wfjKGvCxTvi+6a0E1m9slrAf06MkK7T59lZ0kbDtAV/Ggmyh9qmYLTFauPLpjT731csJKR7Q+Vr/ZPKY1zz/fbGjHSbbs6RqjugRJ14OA1qIIk7gnZWrXw6EKEJtbz/V96Yfo8x1f2zxiE2NZ2ocsLXdp7o/ZmgKDclMJMaTbgUSUvo6Lz3l8sieyYHrRdTE9LuflzMpicRUBiDbrzfTlReMOUFe0kUOdKF2CqomScVImS0rWRUD8rS6bjIiPu0nTGNX2N0gBTs8wVqjWCQJb9piL1bqI2Cnf8c1qqcvSPqTM9xzkQyXEUFo/uQnd5OC7Y8xzkTJZI0784wupunDzXakj+mnmCB2auZawcuiyV7hij6JQms5qN76pTo+eJict3aSkWwBc4pVOOmagpMG2S9Vdm6moJHd9pms11dIqc2BwWRiGCtW2+2jKHiOiXtCyDvGuFRU75Tu+aSx1OiFoquperwW0qsyhZbvnoBgqIYay+Ml16CaHk5fmTd6wBK1mI9fFx2SNSHse287NZXKyTbac1S6s16gRUuqFyCW+QR4r9fyHZq4VJhC4a3M148vXx/2NqB116tI8Xfj6x9h3KXtgcFkoe8C4y33krCnnriwmihTKyO9aUbFTvuPra0nihCBXb42IaHUtou2ID6oklRBDvn5yInPNleWnK9pJxDV82bSoZ9mx5x3catu5uUxOpl2baeF/0OnSxRP7C1mIxBgvdbpsfNEgBIKPGV9cn66EQVgP6OHjlVjxvVOX5unGL/fo/OS+0gcGl4kyB4y73Efu/fOpzl9k7JTP+Ppakrjx2TxWM8YNIj6omlRCDOl2INxCrCssKJtVs5acbzZCmj6WtNaoNYLka5G/Qz+x7dxcJqe0QYvPP7Oc5Sl+fl3qUCOsxapP6+okCfotELj4J1E8Und93/5wlz58e1/sb7ieTh9fv03/uvB3q9uyrJYQEMdFpPi+fwFRIjO0LLFTvpYkU5ZqsxGyYgjxQdVkJHuTucD10tGlXOcJ15vLZBEoc+8n35gFF/IM+k1zfpmAiH6S+ra5nlNXXiGLyOD6yumuL21Ps6z9ncqCXGrAVOF42HHp1cfVpyKKqKO0pFH/VuDTq7BMsHM88VZ8W0FaMHxUujeZC4f37NQXJCxQCBHpdyu2jKAypfKquMTmEG0IAdPw6qoi6/BZ7LIUWiPy3yWayiuI+i8ixmz68qLWSqg7JjeB664vbU+ztP2dygRXhXvQVtYicLHMctbGS/92J3G8sBZoXU6cRYaIUieg2MjDQsklldiEENFoPi/ATCXFkMhgysK44m5xRbd42cROFrNtkW4P7ti6n4vdZprO8+o5fRY7VyHJWUV8s0h8yissdbq9axZ/q7tP05cXtRN4QOsTvq5Qo2upAqKNXbBrf6eyMjvXpvc/W2DbkAybsLPh6jZSNyyHZq5pm49uGquxY6OrZ1REEH6acAHjHOeR1TKIWkugPFRSDGW1FjTCOm0a8xdD3OJq2slnSetUW3T0rBSUfbfDTYY3frmnbZwqzpk1Xdl073STl4uVJCDq1QDKKhx9hYMus0ceMyJ9Cw+ijclblzHzu5d20P/82z2rIJJF6DA3YhXPo60x7bAIO1d0IsVmreHG4NHTVTozeytRhoMo+V4UEYRvcmlzxzaJsgtf/6gVfT6M2vMCeCophmwPeLMR0tbNY7GdtmuXZBUXHzSXRZQ1NX768mLCKtFdW28WmlUMcZPhp9/fSSxI8kSW1eRuu3fq710ytLaENW2bkDSkcVHpMntEb7b/99ic9cPdh5//oxPLxtN16FZFaF51dfKyRhbhDh0GYZcWm7VGjKdJHsgWxfZSh6a+WCCKKLahMr1PWRJM0oQLcM+/yULowyg/LyBOJcWQzRLjEsfB1agZD2u0fetmr4WgqHomnEWB+7kPnCjhJiD58y4m96kvFmj68iI96MT7cNnEhjp5qWO7rRFSEKynzwqBIAJJ8zD16wRFWAtiMUOu2O6TzbWlG2fTM5bHc5iX+8TFHSpfq8vIjnrxPJO1hihZ+VyHOo66Z9bW+iUNs3Ntq5DyCTEwCaGtm+rUHN/kZDE+vGen8TNgdKikGOKsBVzauw7und8stVb4danTm4hcBNEw+aY5UcI1czXtsLRxNquRts2HydLDLXbc2LpU0PbFZPk6d2Uxcb5GWKfNYzVvgbp9PKTJiZaXa8vlGdN9xsfS88FXf01kKYmduo/1zeYOtRUNVKlCdpAp1T5raIAraawxQviaSBNiwPF0Za23kTBdbUREX95s08FdO0b6uQHrVFIM5bED5oqW3V/ulqa6L1dpVVQnzgLnUtEVAVQnMnVx9WnSqsa3ZEmdLqqYHCc6uOByIrddu4yIcSq6ZYSPpefM7C1tujaRf1aX7R6YigYKhiH9Oyvy81QzbERcnmlbg2MZbtPTSuFWsgk1U7hAmn5j3bWILnz9o9PcgyDq6lBJMUSU3RJjqkdUlvTks0f30tQXCzFTd1gPegtpFkyC0tQXTLe4uk7CYkIX905eCNIwiGJypudOHrPlpyvsgt9shL1jqPehOR5SFG204dDFu/k8hz6Bsp9+n0zX1uHyPmTpMO5aomHY4VyJMnITYNN4bh8P6cgrz2mrmcsxQ+KYLpseV0zv70cn9nuV7uAEoe6cv3tph9MzlrXQLhgOKiuGsuJrDR5EVkJRsUhEZteJbcHX9SJzQRYpecSmlKkRpy6+RxWyROvxR9PH9mr/VjcmavV0LhaLw8d65uMisb0PaTuMD6JA6aCqdnMWlXoQ0FoUJa7FNJ4PH6/QwV07tBsZcS6fTY8rs3NtVsC49m+U3x3XIqvbGiF997d7TteYNg4KDBcQQ31iUFkJRcQiZREiaUWhKlLSpPbq6vFsCWu94/jEjPmQZrEUv5fjjOTr0x3TJS6Ei8XirsfHesa5TrjjmpCFPLczVy2KgxCy/Wx66+peXouiXibhqUvzdO7KoragpoxwHX13+g3Wxav7WdaaQlw5hLT3Umcp1WVR+uibPLLSQPmBGNJwZvZWL0W8HgT07msv0PnJfbHPNBshm00WUVAKa0NRZKkx4uP+4Ha4RP7xPjarCRHRkxX/Ipo2siyW3GLDHTNNgKztvvlYz9597QVtVfcaEckj6/o+iO/PtRcRJSsG2UetX01vfdzL2xph7LMu8VVE/bdemyxbaWK9VLF48ZmLTbdxcC2NQpQuDgoMHxBDFH+J1i0FG1P3ahT1JnhZEP3zq88lJv6wFtB/e/sVIirGNVUWsgQe+1RHXositi+Yb7yPi9WkiEUs62LpagHqdFe9LDMypvtmc7Wq13fopR10/f/cj20ksrpTmgUmAmSFE/Z5CwtX93JYDygIzFYgjn5br7kxWouiVELItOnQ1aVy2ZSN2kYW8FReDKkvEZcN8+n3d3piiGvnsWmsRkTDlybvS9rAYzFu8iQeENH4pjo9epqcvE3Hc7VYyIX7XMh7EcsiHH0tQKtRRI2w7r0QurisXC1U9x49pT//y6uJz6d9H2bn2vTw8Uri5/VaQA8fbwSZDyJr07dnXBacn8vI3RIkUwuIHj1Zod2nr/ZtA5dnAoPvpmPqrZe1MXlEG1m49SCI1Wka5TkdrFuwK41r/Q15x839zaOnq/TBV7dodi5b37OyM/XWy886X2/gsoPidrdhveZ9vMmJFn349j5qNRsU0Lop+/iB9bo7u09fpUMz1+jM7C364KtbXtkgeS9i3PFczmOyAOloNRuJMfnD6y/2/r19PFwvACmRZedrK/KXBdFW4uSleW1vt7W1KPHzvM7tClfNWfSMI9r4HuKZTDs3uD6X3bXIO+B3PKxRPQhoqdOliDaEZdHzWNp5RDemvpuOyYkWXXjn1Zh1sdkI6aMT++ns0b3UCOuJchCjPq9Xncpbhlx3XPIEY/qbKtSlSJulxo3bg06X3nv9xVic1vEDvDWCaw6rWil8mpUSFWMSz5KxZqquq1qARLVcm1Vydq4dC8rePFaL/c7nnhZVp8klI4i7r/2Me+HOFRGxz2Ra65VPdp2PhVDEw6gbhn7MY2nmEW5MtzExnCYRaSrGWpbyKKB/VF4MuQb0vvvaC85/U4Xmfmlcgdy4bWuE9OXNdm8nthpF2sqvZ2ZvJXonyU0ZfVL2W0zPubwnuyzlDbjxEtcuj4VPtdzHkit4qdN1arDrc31ZrWtZqiX3M+7FdH+I8g2uVrPrTPFhogCpnFHFuc5Mc1U/5jHfeYQb0y1hLSEA8657lGY8BlV2AfhTeTGk23HViIgCorWItNlktl0amvvp4awkuoBPddGYnWtrLT3icz4TVb9r0aSNITNZlXQuGt2YuQZf2xrs+l5fFtIuwmE96Guwq+375205c6kXJM6vPnOHZq4ZhWu/i4+mhRu7peVurDGxSXjYBEpeIr+fZRdAdiovhtLs3MXvdM1akX3Aw421qdmowNRt21R3pYhaNP3a7ZmeTduYpQm+Nh3P9/qykLb69NZNY31dZGzfP43rxobJamZqSWMTbmUpPmrDJFR0hUsPzVwztr7RCZS8RH6/yi6AfKi8GCJKt3OXq/7CDLqOy1j4pLnKi4ZpURbn4nql5ekK6/duj3s2bbtXbiLmSNNg13R9WUhbffrBM+HRz3fSlGn36GkyCy6sZbNece9BQGS0droI12GYx3yySHXv6eaxmlWg5CXyi4qpA8UAMZSRUU+jdyWLSHCZ4EyWH3miKnpCL8tuL62LRkfevaayIsbx/c8WvOomPd9slMY1ceHrH7Vp27/Zks16lcWFY5qrhmUec33PufeUE9jq+5LHeAyi9yFID8QQyIUsIsFlgtMt/gERvff6i7EdXdETell2e+qYbWuEFAQbDVpNQbMycrVftTji4T07ey0d+m0tcImPkZFjqQYtVmfn2qybbylFDSCZMvXTGxQuGZO+btYiBAru1XABMQRyIatIsE1weVt+0rpSfHZ7eblruOPIrlrVGhLWAgrrgdY6IbMaRVrBUwYLi3rPm+MhRdG6O0yIv6XleMNZl/izIhHjxpF10TW9B67Pm/w5bhyHkdm5tjaOU6VfPe36Za0G+RBEHmbogwcPRjdu3CjwcsCwwmWrDKKLuA1dHZtGWDf2QzJVsg7rAV14J151Oc05XK9VTOZyCrXuupqNkLZuHqNflzpsZ3B1YRD/5mKIfO7nIOLpBv0ccucnSnf/XXF93mw1nIq8xiJRy27YGHRPO9A/giC4GUXRQdvnYBkCuTBMJmFfV4q1CKBmBs7LXWOqn2TLEHvQ6dL82TfZ76BrJSH+zcXquLofBmVZGvRzaLJACZFRhEh0fd5sNZw63VWavrw4MHGQZmzOzN7SNgjmKOMGDQweiCGQC1lMwv22IPi69GwLSHctSiw6ecUW2T5vatAqu2R09ydN+rprq4c8xGCa52LQrgluXOtBQKcuzdO5K4v08PFKr5VIXiLR9Xlzef6WOl2anWv3XRClEdCi/pgrZd2ggcEDMQRyw5RmbOp6ntaC0I+4HyK3BUT9TF6ZJC6iRdd+QTfpuxbis53LhaxiMMtzMcjMKK4sgBg3XVB7HgHers+bqwgeRC2cNALaVH9MoLqV1WOhPAogQqNWUDBiUWsvdbRNINM2+7Qd14SpQaSuCaSLgFE/k7YJpcu1qugatLrEfbgcW3cuF7I0qCUqtglskUxOxBsIu1rSsgZ4uz5vrvd8ELVw0gho23XWArsQmvpiITaPTH2xgKasFQSWIVAo3KJ27sp6XEJaCwJ3XJd4B86VQkQ09flCzIUx9fkCnfinFxI1eGQ4K4zuHGmKe4rjtJc6bCZM2sKhpmOr+Ii5rLE7ZSlhwGGyJsj3Yvfpq07HKzLLzPQ5Coh0xr5B1MJJk6lpel5rz1oqEW2InOnLi/Sgs5E5d+7KYiLjsrsa9eYnUB0ghkChcIvX/eVuz+qSxp3E9ihyjHeQFyxTplh3LaJ/Xfg7ffj2Pu905LzcNbprTSOwTCn6ut8f3rOTrv7174kO966VxonSi8EyF6xzceG5LNYyecSxuD5v6j13Fa1Fp+SnrS6tEhBRI6zRstSQmGhd5Ii0e1vygUuNLhfgghsekFoPCsUUlyJM12lS0G3H9Un/dins9/PMEafjlZXZuXbM6kW03hriwu9fZcdZNzZhfd3vIB+niHTsvEoTFIEtfd/1mRJsHw9p7o9v5n2Zzrgs2LbU9bzujcu1uMwppy7NOwtRjqzvfJmf4SqB1HpQCqbeeplOGgrhpbUg2I7rii1TzAVbgHgZdobTlxdjAoZoXdCY3Iq6sdEVcSyiwvOgs8JM2Fx4pmdK5+Y8e3Rvzlfoh0tFZ1sNn7yeARfrlkt/Ns7S60qzEab+W0EZqqEDdyCGQKFMTrTYqrC1IOi5tFTxYGsBMTnRonNXFrXmbB9Xiotw2j6enBhl15q8wMkuEyJ7h+x+wVXlNVXr9RGV4rN5ir9BZoWZsLnwTIv1xRP7SynwTLi6+/oVz+XiQnVt9rt9PIyVOiBat5j+86vPJTre+96nsse9gTgQQ6AQ5EWxOR5SWAsSlolQ7VCGAAAgAElEQVTVKIqJA9906rNH92YusGdLNQ7rAR15JT4xHt6zMxZQrS4UctbTMO8MfWoRlalJatFMvfUyTX2xELOShfWNbvSmxbqsAk9GFbQ+z0A/cIktUi2LTY3oka1yaqyc/H6nfY7LHPcGkiC1HuSOmvZ+f7lLFBDpsoxl4eCbTq2mMbumlMvoUo3FZbaaDTrxj+uZZHLq7SfXb1t3nL8udUq1M9RZt0w/J9KPTVgPKKzFb6RLk9SRgyvdTfmVVRgEupIVLsUBAsonCNwF1/d+cqJF351+g36aOUJzf3yTTvzTC71SB/UgoOMHNvr7ic99d/oN+vaHu7k8x8P8HFQRWIZA7rjGmgjaSx1jUKRJPGTdadtiUw7NXGPbYZgQu7+y7AzPHt2rtWaY4lVMJQh04zXoJqn94sLXPxrjryYnWnTjl3v06fd3aDWKYgtv2eHav9jKLkTUX+ufS5yTztojil+uRhF9ebNNB3ftSBwnr03MMD8HVQRiCOROmsXPVOemaPFgmljTfJdGWO+lpet+N4idYZ4Bydx4VcUtYCvrQETOC2/Z4L6bKFxoyuIqCzp3rS4AXLVKi/eiOR6mikXMIsDA4IEYArnDLYrNRkhPVtZYF5NuBxrQhuVoEMGm3Hfhur23NDEHvc8E8Qm43xlnvlY03xggLmh1+enKQHpdFYUpjubclUUa3zSmdbO8/9kCEZU7for7bqayAWVz/ZiaG6uo9YbaSx0KawGF9SBmRbV9R927omseO0wxg1UDYgjkDhfgOH1sI1iRW0zkHSiXpdXPiYT7LscPtOjbH+46u9aINir9ljXjTMU3NVj8TM0evL/cLc13ygNTWYf7y122YJ8uYaBsmWW24OQylzwQ+Fhz60GQdOmvRdRshLR18xj7HdV7d+/RE+ooRR7zuD7QPyCGQO7YJszJiZa1cJ3u94PYVanfRVTe/eT6bXq+2aCLJ/Y7xxzIiJYknBWhDLtHn9gJeXGoaSLly/Kd8sBU1oFofYHlGtrKlsEyimAXsVP2jDifDDjuPj3odGn+rL4Qps4K5Ht96vHKLC6rAipQg0LhXnRbddbdp6+ypu1Ws+E1cbhONrbPuVaUTdMNXiUgop8GXPXaJlgFrhWX8/pOutgMzkqX9dimZ4WzDhGtPxemwos2d1RZGMaFWvc82gLAVWTLkNp65NGTFWN9LhPqfCEaxaoJJn94/UU6P7kv1TlAHNcK1BBDIFfU/kWPnq4kfO9iMtBNtERmN5qugq8pnZ6bGN9TJhsXoZO3OCDirQhlWBTzFn95fCeXsU3b8oBrP7J101isuac47v5z32gXRdES4v3PFth7++uz1HUVk2DstzAZ5nYSaWslEa0XXaTAnAGbFlXkTPzpG9bC+JHG6gz8cRVDqDMEckOtUbLU6SYmFDWAWK7vQUS9v9eh293Z6n9wwZSfXL/dy/zhPqce29VtJOqg1HWFlRRWo6i0tUhc67m4uAXz+k4u7VM63VU6eWmeDs1ci93jNMcWzT1FzZ2pzxdo4k/f0O7TVykIiK25NDnRoj//y6vsveUyk7if6+r/fPDVLa/v58sw141S5xbXbLdWs0G/2TJWiBAiWs8ylO+ZqSHsMIzzKAExBHLDtc8Xt3j69HRyOZ7pdxGRt9DhFqqmpnDh5ESL1hysrkJgZCkcWSTqosKl1JvYPh7m9p18gk99BYPLsbtrEd1f7saKiTYbofbemcSkb0G+QQiTMhUNzYpuvFWE5XLJIFCy4nPPhnGchxkEUIPccH15xeLpY8qOiHcpmRZj03FVoePS70jn33/4WJ867mKef/RkhYho4C6xLNj6QD12zLJR0cUG1QzByTpcA7dn59rexyZatxxt3TzGBttywcamQGWdO2wQwmSU6kbJ462raSYLUV+3mi/yPWs2Qjb+aBjHeZiBZQjkhsvLKyadNGX/dS6lgIgO79nJ/s3UWy+zx1WFjm2nPjnRoq2bkvuH7lpE564s0qGZa7T79NWee8ZlN7rU6Rbu7iga2QKiI40FQ/d8fHz9trdYIbILBnGuNMc2HX92rp14JmR0VjfOHbaN6aJe5II5au0kxHj/PHOELp7Yz1pjXd7bLMj3bPrYXu0iLPe6A/0BYgjkhraXVS2g7eNJN4Kp7D9Hq9mg4wdasc9ElPTDy0xOtOi9119MHFcndFzcVQ+YXdz95W5iASOixDGbmkVtWOIwTIiFhrt/vhYMV5drPQjoD6+/aIwJsQkG13P5HD9tjA/nDgsC6rswyaP3X1kxuX/V791shMYefj7o5p3/fmJ/bF7YPh7ShXdeHYlxHibgJgO54VOQzRTLs11TDj+sB/ToyUqqqq7nJ/fRwV07rNflUj/F1YTe6a7S9OXFROG2Ue/flZdrxXU81qKol53DZT8d3rOzl/EmXK0t6RlwPVcQEI3V3CoT+xasFLCtPpa7dPHE/r6nuZe9plBR6L63rZyCC1xD2SqOcdmAGAK54vpiu5T9FxN/czykh4/NtT1sC1peE44tPkZmqdPtXbOwDKTtezQs2CoYu+IqOuVx04lxtTWKcIXJ1jvnGJGI6MI7r2YS++2lDv329FWqBwG9+9oLdH5yX6JgJRcXl/eiOYw1hAaJrdimjdazewjKCcQQGAguZf/l+j62CahfYkK34LoWYet0V2nzWC1RkG+Y4zBU8mrX4CI6deMmPzezc2221g/RhqXGVeD6CBKbwFqNIvr4+m366e5D+vfbDxJiTaaI58O37xxY5+zRvc6bIZlResdHFYghMBDycKkJ+j3RqAuiT5HFB53BuDv6SR4WDM7K41ppWlT2tQVF/7rUSZxLWCK7a+6NOlVcBdZ3f7un/Xk9CGgtipz6YqV5ftK68aqOGBsfd9n28ZDOHt2LcS05qEANBo6Y3LmYDlOF41ZJxIS6QC0/XdFas8pQWboKmCr7ynD3Iw/BIR/DN09NV4l6dq6tddOIqtBE7hY5rt1NGdrAlBUXd+bWTXUK67WepRhCaPCgHQcoFdzicmb2Fn1y/bZ2YpYneZe2AGWKgRjmVgZlIOu9/O3pq9bP9PN+vPTBX7xS99Wu6Wrsk+7zT1bWnJ83U2uZqbdeLs17VBZc28AcP9BK3CfX56xM89coATEESgMnDI4faLFCSNBshDR/9s3cmqj6XHOeloFRm9yK/G553EubGOq3RfHM7C1tJuShl3bEYoaI9L2xfBuNCkyWL+6dTLuYjzKceFTdmVxfRZtF2LWHIvDHVQwhZggUDhef8On3d6wT/FKnS2dmb1ljRfKMgdAFl059vkDnriz2OlebqgXL7RhGcQEpOvg2j3vJVfYV4roouOdBLGiffn+HVqOol012cNcO+t9/X+x93+azDumqKyxtpywu3o6L2UMskf4ecuO4FkUxt2La0hmmHooHd+2ozNgPEoghUDjcRODqNpCtR9zCm2e7Am3Dzmc9qeRruPHLvdguehQyclwsPkUvmHncy+lje2nq84VYEHRYC2j62N7M1yfQtQsxPQ/nJ/fFdvk6a4Dq6spKLQi0rWLENak/H/U6WDY4ob+NEddqFmvaOlu2HorDOp8ME6hADQqHmwhcuroTuXWqN3UBt7VFUHGZ+IVla1i7eutwrZqcd58s9f7oGt8SrS/srvdwcqJFF37/aqx68oXf51fVVzdWn1y/7fU8cKKSey/c3pY4q1Hk1e7F9B6NMuIZPHlpPlMF8LQtTEzjWxUhOmgghkDhcBPEu6+9kLoHkDpBcOc4vGend1sE14mfs2y1lzpD2WuMW5xPXpqnlz74C52Z3ShSqMNnwRSLz29PX6VTl+Zj9+fh4xUK68mlfzWKvFpbmFouZIVza+jwFY+6HnyNsE7vWdqOcPgI9FHrR2Zjdq5NE3/6hk4+ewY5lpa7Tq1JfFqYyJsA0bBZx6gL0bIANxkoHFNNIblNRiOs0bJjh3N1gsgzBsK1RowpqLVf7rI8A5lNO1BRJJAoe5Vp1RWhjmF3LYplU+nSmDvdVXr/swUich/jfo2VCte7jEvPtmV0mUpNZL3evIpmDgM+9cF8Cm66fE7NouWKto6yEC0bEEOgL+gKFR6audabcEUhwmXNpK2KDm6CyCsGQl0QtjVCevR0JZbdE9YCWiOi1TW9HOpH0GnegcwubSk+/f5OL+6FWzBtosOlMepSp9sLdN7NZIYJFxCR+fvOzrVp+vJibMEpaqzUZzV4dq5DM9diQfcffHVLK4QCIjq8Z6fx3DoxGtYD2rppjF1UZUFmuz+jGviv4tqgN29BMjvXNmbRimeoLDXUqgLEEOg73CLOTUxiYkizU00b0KgTby5FFWWK9vXnHcjsYhETCzi3YLoINJdxCZ4di4hYCwqR/fuadv+y+8jXEsJZx44faNG3P9yl9lInJozkcTAtwhERXfpfd+jSv93piW91DE3WG933DWsBLT9dod2nryaqa49C0H9aXJ7DvASJWrDRlDoi5jsUZ+0vEEOg75gCRzm3QdqJIa/Goeriz1krZIr29ecdyCwvspyFyBb0zt1b2aXlYoGKiGj68iI9WVlzaqmhy+wSosREe6kTa63gKg5s7iSdK0uIL9v96WqsjeoYmiytzfGQNo/V6EGn27NqCuGuE/BVS50XmJ7DPGsrqQLVJYsWQdP9B2II9B1b4GieTUyLioGwLej98PWntXqZEIssVyTw3ddeMP696d4KkeEak+XS/JaIaFsjTFijdNfuims8ksmdZBKqLmJQh+oWlNvYyFao+8vdXsG+b3+46zSOVVx8ueew2Qhp+lh+LTRc3XEyCJruPxBDoO9wi0FRrQCKiIGYeutlOnVpXmvurgdBXyr25mX10sEVCTw/uc8Yc2Ja6IUFQlj5bH2eXGiEdQoCyrU2D5F7PBKHSai6ikEdsmvPFIQuCva5jmoVF99+BYv7Cs2ACEHTAwDtOEDfGZW+XVzLh6zNLm2Na3WfzTqZux7Hdu9sGTpcA1LdMbeENa1bR22BwInSPEjronUZJ51Vh0jfjkMmILdgdyJiXc8yw/juDRNpsv9+RrPc3EA7DlBahjF9VycWWobdf1qRwsUXcLEseVi9fLLSbEHb4vP/9bN50iXa6SwQ4m/kjK8tYY2OvPKcU58sU4xTVvKIv7K1atE9K0RE73+2oBUyzz9LJnBB53oWmWcPOt2hePeGnam3Xk5UQzeRppYUyA7EEBgIw5S+y4kFrqnl4T07aeqLhVg20NQXbjVxTPEFaqBrXlYhn6w016BtYb2RCeuB0fz/ZGWjxtT95S59ebPdy84yfcc0bqft4yE97tpbX+QRf5Xlc5wb1FUA1oMglpyAdO3+MznRonNXFq3Zp0T2dwQUB8QQABY4sfDtD3fpw7f3JQTJuSuLCRdHdzWic1cWrYuQbccvfp9njSGfrDTOPbOtsdFC48LXP2p3wVs3jbHXZhpjm5tKZ4WRs8l0darOHt0b+xs15Vx8LsvCxGW4ccJOJ251z5f4GxcBKCxLwkIEITQYlhyE0PbxkM4ezS9wG/gBMQSABZNY0O3qTzKFHl12hrZYEGGpyLPGkE9WGmfyf/R0pdcQlBuvB4asJu5v1IKFKqqAuHhiv3dclSyIoohycR/pxKqc4aaKV07cfvj2Pq0YdCmDoFLVFPoyYEoaQT2hcgAxBICFIlLYifSLtMnlI1sq8qwx5JOVxpn8u6tRb6FNM14mESgLByLemsN9jhM2akuE+8tdaoR1raDyxSWdWhYnLvWZVIQQ9wnQrWIKfRkoMvMT5AMatQJgwbd5ZbOh77ou/5zrEE9EvUaPRBtFDtWGj3l2F5+ccG8uScSb/MVCaxovtUO9qDKt+xuZTneVpi8vxsbs/nI3YaHSfU7X2JVrieDT1NSEq+gQn7PVZzI1pfVZUKuYQl8GfN8x0H9gGQKVwzfw2Df7bfrY3oQrKawFNH1sb+/fJjeXS4f1Qe40bZYfbryIyBrnZHL7uBZh1H1OdRFd+PpH7y7zPrimvosxc6nPxD0TkxOtRO81HbBEDJZhShqpIqgzBCqFrv6LqNYrCg3a/t61Ho/pc7tPX9Uuxj41ivKsMcTVxCGyixr586bzc+4cNW4iTV0WF+Sx5cZfdz1pcOmInrU+k+18SKEHAHWGANCis8iIar0Hd+0wLhamDC5xbFk0mBbUPOKQ8tppclYq0RtMF9RrynLiMMU5ycJuWyOksB7EMvJMRRhdkcfW1HU+D+uJKcONqztEZK4t5Hs+Xbaa6F8GcQRAHFiGQKXIYhHgLBbNRhgTDUR2S0mZqnCbxkRHWssJN366mj9hLaDfbBmjpeWu0SLFoUunl8c2q4WwKHytdK7PSpmeNwD6CSxDAGgwxWa41vhRcYlRUSlTFW7fxqEi3d33urk4pyhK9hbrrkU0vmmM5v74ZuI48phx1x3RumjjrrEM429yc6aJtzKRZykGAEYRiCFQKUy9rGyuCF/RoIon3eLna2HJK05IhhMpnFsqIOqNg65ejqkFxY1f7sWavx4/0KJPmA7zOvGpugZd45B0DDKgVedynfpigaYvL/ZifOQU/0Mz1zKJmTxLMQAwikAMgUohFmQ1rTqsBbT8dIV2n76aW9sHWVzNzrVTt+iQj6EuoKcuzdPJS/OxNgtcryuTSNH9nijpllLdT0R8J3WdUPryZjtWFfnj67cpcBg/dRzkWkNhLbBWjlZjkoKAYi64fosinaWmuxr1rIzq2GUVM0XVygJgVEDMEKgk6uL46OlKImBXF08h/13N0BFc/fuJP32jtbJsHw+1riAdtiyrRljX9ksL6wFRRAnBYItpEmnucl8r7vymTurCSuOTJcZdn1ooUXw/U9aULVNrELEzrnFatrGzWcDk+2iLowJgFHGNGULRRVBJJida9N3pN+inmSO0dfNYopcYV3xP/ju1EamMushwWVA+2VE2K0Cnu0ofX7+ttTjoihNyxQXlgpBE8b5WXEdtUyd1W2FBlWYjZIWorlBidzWirZvH6OKJ/UREdOrSfKygo60adF6FFn1wtciI+CxORB7es5P9W/U+RkQ9KxyK/gEQB24yMNK4xNj4uiDEMTkpxLl9suIbs2RDXmhl68/y0xU2PsVU7JErmOhSWFBm62Z9Q1fTmAu3ks5F5yLCiqhrZMLV5SrHZ+n49oe77O+4MhLohwVAEliGwMjCtbxQWxv4tLZQd9s6IqKepUHUduHgWnfosLWs8EVeaIW7r73UYa1VojEt11bA1oZj+emK03UJkaa27DCJmnoQsALOxQoTEBlbXuSNPI4mbK4005iYer0BAOJADIGRxZROLOPTe8ylASfRRiFBk3BSW3TYUBdQVwtUWA8orMU/rQuEtiG32xCuQrl1CCeUiNYDq11dgkKkqQKWEzUBERu79etSx0lEygK2X4hxtAkiEyahJ/rauf4cgCoDNxkYWVzdX641Z2bn2s676uebDaNwajHnsCGng8/OtdmKxfUgoLUoYrPJfK0DAZnjU3TXJ9ClhRMRBQGReummbDWda0kUSvz2h7usi069v0X2JEtD2vNygl24cTmByP0cgCoDMQRGFp90YlvNGWHlcUEsUqcuzWt/HxDlErPBlQngsoRc6vNwRET05c22tWWJDnaxj4g+OrHfSaQJFx2RXrRyFaWFy02u6cR990GlmXPfu6nJchRsHw/p7NG9MWHMZY2pZLFEATCqQAyBkSVNZ3cu4Npk5eFSu20BxVkRdXvkhS8gouMH7MUEfWsmEcVdjD6FH02i1LWIouyi051L7XovCwK1Zk+a56JIuOuZPraX7UY/vmksJoTkvzcJIXSuB0APYobASCJETae72ouRsKUTmwKuTa6MC++8SvNn30zE0PjEIqWByxYyZRgJJidadPxAyzvzrb3UoanPF2JjNPX5gjH42GccsoyZHINjKgxpCgIfBKbreaARQkRxa5trHNugvycAZQaWITByqDtlESPx6Ik5m8kUcG0qKDio/mNZqxJ/+8Nd7yDqgChRs6i7FtH05cVU46CzxH349r5MY8Z9/7Q91foBZ/HinruI1gtQnp/c53S/kU4PgBmIITBycDvlpU7X2EfLloqsq+Brs1gU2f8qTYsF+Tv7CqFGWGctEDpXjoxuHHTtRT746hZ9+Pa+TAs3Ny6mnmplxeTO/PhZTzdbQDxcYwDYgZsMjBymnbKw9OhcYjaXkSoeNo8N9vXxdSmp39lEsxHSH15/UZsmnxeupQ9ErSa17hAHl0pvcp2lOU8/EC40jk+/v2MsHcBV8wYAxIFlCIwctp1ye6mjTUkX7QpcLSaqpakfqNas4wda9O0Pd51cP66xJURET1bW6OCuHXR+Mr4Qn7uyyPZY88XFzcdZj4j4MTcFU6uI5yTNefrF5ESLTjKZiatR1Ls+3b15srJW+PUBMArAMgRGDpcie1ytFdGuwDWwuJ99rXTWrC9vtmnqrZcTwds6fOrZcN/r7NG9641fJcJ6QGePuhWPlK0vNab4n+zmc7UeqZiCqWVEcH3a8/QLWwHFyYkWjW9K7m073VV6/zNzgDsAAGIIjCAiUyoNItD0p5kjzvVY+lWsL+uC7ZvSr/tekxMtuvDOqzH32YV3XnWynqhiTidIVTdf1iBx2+fENWQ9T9G8+9oL1p9z17oaRdo2NACADeAmAyOJS3q5iroQu9bi6VexvqwLtm9tIe57pQ0K59x0umrZIuur9qyBrOu16T5ncpkKwZsmGL2fCHflp9/fodUoonoQ0LuvvRBzY5q+qxDNg3b5AVBWYBkCI4mrQKgHAVtrRq3/sn08TPT4csnUOTN7i1764C/029NX6aUP/kJnZt0qWav4NJTVISxmLr2pishA4u7JWhT13HxE5G09MmFymcrHKbomlA9cIPf5yX30tw//C310Yj/9p21b6JPrt2O/t7mHy2LlAqCMwDIERhKX/lu6thW6ujdymveZ2Vux3bmt2vOZ2Vu9FGii9cVd/FsNTraRtXKyqFht602ltnrICxfri6v1yPXa1GDq+jNLk9obruiaUK7YArldAr25fnVlsXIBUEYghsBIYnMJ1YNAK4RsC5EsJlajyNqv69Pv77A/9xVDtgWbayUiyFNo2M6lw0XM2axHaXB16xVZE8oVU1wY1xZG/r24/jK1GwFgGICbDIwkwsWlS/luhHX6878kg35tAcppApj71Tn8zOwtOnVpXttKRGAKsJWx1dvRZbWdvDRP+899YwzSdWmDkdUVqFLG2kEmTHFhs3NtYyNbQdnajQAwDMAyBEYWsVN2sWK4LDSmCtW7T1/VHrvOBAC7xO2ocJYrXed6orjFYHauzQYjE9FGn7EvFoiijZYbOjeMa4VvHTbrS55NVMtcO4iD7WA/HvauXce2Rlz0l8HKBcAwAcsQGHlEzRmuFo9YNDmEVcIkYDhrjEtKtCucZerT7++wtXSEReGDr245WaO6q1Gi95hq/XKp8J2WPK0aZa8dpIML5I4iMmYBptDWAAAJWIZA5TFVZpatEi5iQk1hFnFB/+P72yQ0RiOs0cFdO7yv09XNJfN8s+FVedrl3Lbg9KxZS3lZNcpeO0gHFxd2iqlALbi/3KVDM9dK1XwWgGECliFQeUyLo2yVSFuE8eCuHbR5bGO33+mupSqCx8XNcBargNYtDXks/vK5D+/Z6fzZInCNA8o7/qhf6CyZLtess0wCANyAGAKVh1tohPgRC+/y05VEnSGX47m4a1yClu89epI4V0BEr//n7QnXSkBE773+onEh3T4eWtuWCJafrtDsXLuXUWeiyKwll0Bx+TrKUjsoKy4tZojWn6tzVxb7cEUAjBYQQ6DycIvmb/+hEVt47y93iYL1TuABrf9X16fr0ZOVmKixuWt02VnyAj8716apzxeo00023YyI6N9vP6DjB1qxOJuLJ/b3XHTc9zt7dG8vPsfG/eX14OhzVxaNLrft42FhbprZubYxUFxllLKq5O9CZI5fu7/chXUIAE+CyCPF9+DBg9GNGzcKvBwABoOacXZ4z07twku00b9M/bvmeEgPH6/EApAbYZ02j9VoqZPs9C6Oc2jmmjYGx/Z77ppcvp+wjpiu3RddEUsTvrWKTOMQEKWuQzSsmMbD9jwAUBWCILgZRdFB2+cQQA0AJYN2D81cYzO02kudXu8seRE/NHNt3Xok0emu0pawRo2wzqaL2yxHLjE/ts+oZQZOXpqngKj3He8vdymsB9RshPSg0zV2etehK2JpIk3au+k7lj0OqAim3nqZTjKB1WUOEgegjMBNBoAGkyUmePZ71aXFLUBLy12ju8YW6Ouy0Kuf0cUgye44IkoInu5qRFs3j9FPM0dY11mzkYwz4opYmpi+nHS32dLeuXEQgeJVY3KiRc1GsqgoUTXFIQBZgGUIAIXZuXbMaqLCxayYem/p0sWFlaa91EmcT20iOvX5AuvCUoOCOavLlrBmTbEXgo4rfjh9bC8RZevhNTvX1roN5fPr0F2THCheRaaP7UXrDQByAGIIVB41dmX56Yq3m+jXpQ5dPLHfeWFSBUtE1BNEXBPR6cuLPRFRC4jWouRnifjsNZdaQ8KiYOuDlkV8mKw/aiVlmbI0Uy0TGBMA8gFiCFQanRXFRMti/SGKd0iXXT82wSILoQtf/0inLs3HFjfTAicLurQh0KpwK6qlg8n6Y6ukjDYTSTAmAGQHMUOg0vhUZxZCxVS7ZnKi1fuMqAzt0zRVfNaljo5ATc3n0MX7CO3Rz7RzUzzL0rLefQYAAEUCyxCoNK5ZN0Lw6NwSh/fsjFlyHj1ZYYOD5aBpnYVJWJNMf6viIugCWm+k2myEtCWs0dJyN3W8T1aXjGgvoRNuCPwFAAwCiCFQadgu4Y2Qtm4eY+NlxP/7uNlk4XV4z076+PrtxGe4PmMm0WYTdHJw9lKnS42wThdP7PcWMXl1gZ+caNGNX+4l6jgh8BcAMCgghkClMWVNuSzwPm6255uNWAaZjnoQaAO8qtQAAAUPSURBVAWRyWLCCTqRHq/+zmZp4jC1FfE91vnJfXRw1w4E/gIASgHEEKg0abNxbKJGpRHW6fCenQnhpbIaRcYCjTo4QWfqdq6zJtlcYHl3gUfgLwCgLEAMgcrjuyir7iId28dDiiLqpcJvCWv05c3/q+0vJiNnk7mKM5Og4wSbrkijzQVmqqMEAADDDMQQAJ7YXGONsE5HXnku1t1dbdPB/Z0tjZ6z3nB/Y7Ia2b6T6gJzPRYAAAwbEEMAeGJzC20Ja3T1r393jiUi0hdPVEkTwOzqBnRxgaHAHwBgVIEYAsATzl0kcLECyXzkmNnlYr0xWY5MuLrAEOcDABhFIIYA8ETnLkrL9vEwc7B2e6lDu09fpeZ4SA8fr/R6mPmkvsMFBgCoMqhADUAKtoTZXx0RW6R2l5dRO81zRLRukVKbuXKd4NWu9kREH769j1rNBgXU34rUAAAwaGAZAsADXSaZqcO9zB9ef5G+/eFuz4X1239oxAoP6iw5PnWMONR4IC726MO399F3p9/IdC4AABhGIIYA8IBrsGoTRK1mg85P7uv9e3aurW1JocYApa3hI6PG/eRZPBEAAEYBuMkA8IATJ6LjPNFG81MBl8rOiSf5HFwNn1az0TufCd258y6eCAAAww7EEAAemMTJd6ffoJ9njtDFE/t7QkU0Xr3w9Y9OXevVc0y99XKi07wQOLrfhfWAmo3QGPfTHA+9vhsAAIw6cJMB4AHXYPXwnp29/xfiQ43LOXVpnm78co/OT+5jU9mDZ8c6NHOtF1t0/EArFmuk1vbxqfszO9emh49XEj8P6wEyxwAAlQViCAAPvv3hrtPPudiiT67fpoO7dmhT2QMi+t1LO+jLm+2YiPryZpvN7PKt+3Ph6x8TGWdERFs3jSFeCABQWeAmA8AD13gbU2yRCFRWU9kvnthPP/9Hhw1uzgPuuh50/ApFAgDAKAHLEAAeuFZqNlWpFoJEZ9Xx6TKfBjRbBQCAJLAMAeCBKaBZ/ZyaVSYwCQ/ud3mJFdfrBwCAKgExBIAHOveWLp5ncqJF773+olOavUzRYsX1+gEAoEoEUeRSO3edgwcPRjdu3CjwcgAYLbjGqXn/DQAAgCRBENyMouig9XMQQwAAAAAYRVzFENxkAAAAAKg0EEMAAAAAqDQQQwAAAACoNKgzBMCIgQBsAADwA2IIgBFidq5NU18sUHd1PTGivdShqS8WiIggiAAAgAFuMgBGiHNXFntCSNBdjejclcUBXREAAJQfiCEARoj7y/oeY9zPAQAAQAwBAAAAoOJADAEwQjQbodfPAQAAQAwBMFJMH9tLYS3eES2sBTR9bO+ArggAAMoPsskAGCFExhhS6wEAwB2IIQBGjMmJFsQPAAB4ADcZAAAAACoNxBAAAAAAKg3EEAAAAAAqDcQQAAAAACoNxBAAAAAAKg3EEAAAAAAqDcQQAAAAACoNxBAAAAAAKg3EEAAAAAAqDcQQAAAAACoNxBAAAAAAKg3EEAAAAAAqDcQQAAAAACoNxBAAAAAAKg3EEAAAAAAqDcQQAAAAACoNxBAAAAAAKg3EEAAAAAAqDcQQAAAAACpNEEWR+4eD4C4R/VLc5QAAAAAA5MauKIp22j7kJYYAAAAAAEYNuMkAAAAAUGkghgAAAABQaSCGAAAAAFBpIIYAAAAAUGkghgAAAABQaSCGAAAAAFBpIIYAAAAAUGkghgAAAABQaSCGAAAAAFBp/j9KiWhhynI6pAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.figure(figsize=(10, 10))\n",
    "plt.scatter(item_tsne[:, 0], item_tsne[:, 1]);\n",
    "plt.xticks(()); plt.yticks(());\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Getting the name of the movies (there must be a better way, please provide alternate solutions!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/lelarge/anaconda3/envs/pytorch/lib/python3.6/site-packages/pandas/io/parsers.py:709: UserWarning: Duplicate names specified. This will raise an error in the future.\n",
      "  return _read(filepath_or_buffer, kwds)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>item_name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>item_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Toy Story (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GoldenEye (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Four Rooms (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Get Shorty (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Copycat (1995)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 item_name\n",
       "item_id                   \n",
       "1         Toy Story (1995)\n",
       "2         GoldenEye (1995)\n",
       "3        Four Rooms (1995)\n",
       "4        Get Shorty (1995)\n",
       "5           Copycat (1995)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(op.join(ML_100K_FOLDER, 'u.item'), sep='|', names=['item_id', 'item_name','date','','','','','','','','','','','','','','','','','','','','',''],encoding = \"ISO-8859-1\")\n",
    "movies_names = df.loc[:,['item_id', 'item_name']]\n",
    "movies_names = movies_names.set_index(['item_id'])\n",
    "movies_names.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "item_bias_np = model._net.item_biases.weight.data.numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "movies_names['biases'] = pd.Series(item_bias_np[1:].T[0], index=movies_names.index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>item_name</th>\n",
       "      <th>biases</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>item_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>0.133696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GoldenEye (1995)</td>\n",
       "      <td>0.098708</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Four Rooms (1995)</td>\n",
       "      <td>0.098661</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Get Shorty (1995)</td>\n",
       "      <td>0.120266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Copycat (1995)</td>\n",
       "      <td>0.108264</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 item_name    biases\n",
       "item_id                             \n",
       "1         Toy Story (1995)  0.133696\n",
       "2         GoldenEye (1995)  0.098708\n",
       "3        Four Rooms (1995)  0.098661\n",
       "4        Get Shorty (1995)  0.120266\n",
       "5           Copycat (1995)  0.108264"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_names.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1682, 2)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_names.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1653, 2)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "indices_item_train = np.sort(item_id_train.unique())\n",
    "movies_names = movies_names.loc[indices_item_train]\n",
    "movies_names.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "movies_names = movies_names.sort_values(ascending=False,by=['biases'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Best movies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>item_name</th>\n",
       "      <th>biases</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>item_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>313</th>\n",
       "      <td>Titanic (1997)</td>\n",
       "      <td>0.159590</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>408</th>\n",
       "      <td>Close Shave, A (1995)</td>\n",
       "      <td>0.155670</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>302</th>\n",
       "      <td>L.A. Confidential (1997)</td>\n",
       "      <td>0.155401</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>272</th>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "      <td>0.155322</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>269</th>\n",
       "      <td>Full Monty, The (1997)</td>\n",
       "      <td>0.153195</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>242</th>\n",
       "      <td>Kolya (1996)</td>\n",
       "      <td>0.152459</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>316</th>\n",
       "      <td>As Good As It Gets (1997)</td>\n",
       "      <td>0.151834</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>320</th>\n",
       "      <td>Paradise Lost: The Child Murders at Robin Hood...</td>\n",
       "      <td>0.151096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>Star Wars (1977)</td>\n",
       "      <td>0.150199</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>304</th>\n",
       "      <td>Fly Away Home (1996)</td>\n",
       "      <td>0.149282</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 item_name    biases\n",
       "item_id                                                             \n",
       "313                                         Titanic (1997)  0.159590\n",
       "408                                  Close Shave, A (1995)  0.155670\n",
       "302                               L.A. Confidential (1997)  0.155401\n",
       "272                               Good Will Hunting (1997)  0.155322\n",
       "269                                 Full Monty, The (1997)  0.153195\n",
       "242                                           Kolya (1996)  0.152459\n",
       "316                              As Good As It Gets (1997)  0.151834\n",
       "320      Paradise Lost: The Child Murders at Robin Hood...  0.151096\n",
       "50                                        Star Wars (1977)  0.150199\n",
       "304                                   Fly Away Home (1996)  0.149282"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_names.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Worse movies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>item_name</th>\n",
       "      <th>biases</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>item_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>907</th>\n",
       "      <td>Vermin (1998)</td>\n",
       "      <td>0.028771</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>852</th>\n",
       "      <td>Bloody Child, The (1996)</td>\n",
       "      <td>0.028060</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1486</th>\n",
       "      <td>Girl in the Cadillac (1995)</td>\n",
       "      <td>0.027855</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1654</th>\n",
       "      <td>Chairman of the Board (1998)</td>\n",
       "      <td>0.026460</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1494</th>\n",
       "      <td>Mostro, Il (1994)</td>\n",
       "      <td>0.026367</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1290</th>\n",
       "      <td>Country Life (1994)</td>\n",
       "      <td>0.026174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1621</th>\n",
       "      <td>Butterfly Kiss (1995)</td>\n",
       "      <td>0.024711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1659</th>\n",
       "      <td>Getting Away With Murder (1996)</td>\n",
       "      <td>0.024535</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>599</th>\n",
       "      <td>Police Story 4: Project S (Chao ji ji hua) (1993)</td>\n",
       "      <td>0.023439</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1146</th>\n",
       "      <td>Calendar Girl (1993)</td>\n",
       "      <td>0.023177</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 item_name    biases\n",
       "item_id                                                             \n",
       "907                                          Vermin (1998)  0.028771\n",
       "852                               Bloody Child, The (1996)  0.028060\n",
       "1486                           Girl in the Cadillac (1995)  0.027855\n",
       "1654                          Chairman of the Board (1998)  0.026460\n",
       "1494                                     Mostro, Il (1994)  0.026367\n",
       "1290                                   Country Life (1994)  0.026174\n",
       "1621                                 Butterfly Kiss (1995)  0.024711\n",
       "1659                       Getting Away With Murder (1996)  0.024535\n",
       "599      Police Story 4: Project S (Chao ji ji hua) (1993)  0.023439\n",
       "1146                                  Calendar Girl (1993)  0.023177"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_names.tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. SPOTLIGHT\n",
    "\n",
    "The code written above is a simplified version of [SPOTLIGHT](https://github.com/maciejkula/spotlight)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you installed it with: `conda install -c maciejkula -c pytorch spotlight=0.1.5`, you can compare the results..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "from spotlight.datasets.movielens import get_movielens_dataset\n",
    "\n",
    "dataset = get_movielens_dataset(variant='100K')\n",
    "\n",
    "from spotlight.cross_validation import random_train_test_split\n",
    "\n",
    "train, test = random_train_test_split(dataset, random_state=np.random.RandomState(42))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = FactorizationModel(embedding_dim=128,  # latent dimensionality\n",
    "                                   n_iter=10,  # number of epochs of training\n",
    "                                   batch_size=1024,  # minibatch size\n",
    "                                   learning_rate=1e-3,\n",
    "                                   l2=1e-9,  # strength of L2 regularization\n",
    "                                   use_cuda=torch.cuda.is_available(),\n",
    "                                   num_users=total_user_id+1,\n",
    "                                   num_items=total_item_id+1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0: loss 13.084344199941128\n",
      "Epoch 1: loss 7.23193363901935\n",
      "Epoch 2: loss 1.7397987254058258\n",
      "Epoch 3: loss 1.065873310535769\n",
      "Epoch 4: loss 0.9410984697221201\n",
      "Epoch 5: loss 0.8957061956200418\n",
      "Epoch 6: loss 0.8725117656249034\n",
      "Epoch 7: loss 0.8602462101586258\n",
      "Epoch 8: loss 0.849739787699301\n",
      "Epoch 9: loss 0.8396177744563622\n"
     ]
    }
   ],
   "source": [
    "model.fit(train.user_ids,train.item_ids,train.ratings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "\n",
    "from spotlight.factorization.explicit import ExplicitFactorizationModel\n",
    "\n",
    "model_spot = ExplicitFactorizationModel(loss='regression',\n",
    "                                   embedding_dim=128,  # latent dimensionality\n",
    "                                   n_iter=10,  # number of epochs of training\n",
    "                                   batch_size=1024,  # minibatch size\n",
    "                                   l2=1e-9,  # strength of L2 regularization\n",
    "                                   learning_rate=1e-3,\n",
    "                                   use_cuda=torch.cuda.is_available())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0: loss 13.094079403937618\n",
      "Epoch 1: loss 7.267294666435145\n",
      "Epoch 2: loss 1.7425648668144322\n",
      "Epoch 3: loss 1.0705132137371014\n",
      "Epoch 4: loss 0.9392573350592505\n",
      "Epoch 5: loss 0.892018311385867\n",
      "Epoch 6: loss 0.8656685488133491\n",
      "Epoch 7: loss 0.8464508524423913\n",
      "Epoch 8: loss 0.8351905655257309\n",
      "Epoch 9: loss 0.8227788345723213\n"
     ]
    }
   ],
   "source": [
    "model_spot.fit(train, verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "item_emb_spot_np = model_spot._net.item_embeddings.weight.data.numpy()\n",
    "item_bias_spot_np = model_spot._net.item_biases.weight.data.numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "movies_names['biases_S'] = pd.Series(item_bias_spot_np[1:].T[0], index=movies_names.index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1653, 2)"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "indices_item_train = np.sort(item_id_train.unique())\n",
    "movies_names = movies_names.loc[indices_item_train]\n",
    "movies_names.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "movies_names = movies_names.sort_values(ascending=False,by=['biases_S'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>item_name</th>\n",
       "      <th>biases_S</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>item_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>302</th>\n",
       "      <td>L.A. Confidential (1997)</td>\n",
       "      <td>0.155696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>313</th>\n",
       "      <td>Titanic (1997)</td>\n",
       "      <td>0.155077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>269</th>\n",
       "      <td>Full Monty, The (1997)</td>\n",
       "      <td>0.154634</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>315</th>\n",
       "      <td>Apt Pupil (1998)</td>\n",
       "      <td>0.149982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>408</th>\n",
       "      <td>Close Shave, A (1995)</td>\n",
       "      <td>0.149330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Usual Suspects, The (1995)</td>\n",
       "      <td>0.149262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>272</th>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "      <td>0.147844</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>427</th>\n",
       "      <td>To Kill a Mockingbird (1962)</td>\n",
       "      <td>0.147753</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>Star Wars (1977)</td>\n",
       "      <td>0.145862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>Silence of the Lambs, The (1991)</td>\n",
       "      <td>0.145471</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                item_name  biases_S\n",
       "item_id                                            \n",
       "302              L.A. Confidential (1997)  0.155696\n",
       "313                        Titanic (1997)  0.155077\n",
       "269                Full Monty, The (1997)  0.154634\n",
       "315                      Apt Pupil (1998)  0.149982\n",
       "408                 Close Shave, A (1995)  0.149330\n",
       "12             Usual Suspects, The (1995)  0.149262\n",
       "272              Good Will Hunting (1997)  0.147844\n",
       "427          To Kill a Mockingbird (1962)  0.147753\n",
       "50                       Star Wars (1977)  0.145862\n",
       "98       Silence of the Lambs, The (1991)  0.145471"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_names.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>item_name</th>\n",
       "      <th>biases_S</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>item_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>711</th>\n",
       "      <td>Substance of Fire, The (1996)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1634</th>\n",
       "      <td>Etz Hadomim Tafus (Under the Domin Tree) (1994)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1567</th>\n",
       "      <td>Careful (1992)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1339</th>\n",
       "      <td>Stefano Quantestorie (1993)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1340</th>\n",
       "      <td>Crude Oasis, The (1995)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1447</th>\n",
       "      <td>Century (1993)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1373</th>\n",
       "      <td>Good Morning (1971)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>852</th>\n",
       "      <td>Bloody Child, The (1996)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>857</th>\n",
       "      <td>Paris Was a Woman (1995)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1575</th>\n",
       "      <td>I, Worst of All (Yo, la peor de todas) (1990)</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               item_name  biases_S\n",
       "item_id                                                           \n",
       "711                        Substance of Fire, The (1996)       0.0\n",
       "1634     Etz Hadomim Tafus (Under the Domin Tree) (1994)       0.0\n",
       "1567                                      Careful (1992)       0.0\n",
       "1339                         Stefano Quantestorie (1993)       0.0\n",
       "1340                             Crude Oasis, The (1995)       0.0\n",
       "1447                                      Century (1993)       0.0\n",
       "1373                                 Good Morning (1971)       0.0\n",
       "852                             Bloody Child, The (1996)       0.0\n",
       "857                             Paris Was a Woman (1995)       0.0\n",
       "1575       I, Worst of All (Yo, la peor de todas) (1990)       0.0"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_names.tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Further Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:pytorch]",
   "language": "python",
   "name": "conda-env-pytorch-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
